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Improving  School  Children’s  Mathematical  Word  Problem  Solving  Skills 
through  Computer-Based  Multiple  Representations 


Tufan  Adiguzel 
Texas  A&M  University 

Yavuz  Akpinar 
Bogazici  University 

Abstract 

Instructional  resources  that  employ  multiple  representations  have  become  commonplace  in  mathematics 
classrooms.  This  study  will  present  computer  software,  LaborScale  which  was  designed  to  improve  seventh  grade 
students  ’ word  problem-solving  skills  through  computer-based  multiple  representations  including  graphic, 
symbolic,  and  audio  representations.  The  proposed  presentation  will  illustrate  the  design,  implementation  and 
validation  of  an  interactive  learning  environment  (ILE),  LaborScale.  This  ILE  is  based  upon  the  principles  of 
computer  based  interactive  problem  solving  environments,  which  connects  different  types  of  knowledge 
representation  forms,  and  aims  primarily  to  assist  students  as  they  explore  symbolic  representations  used  in  word- 
problem  solving  process. 


Problem  Solving 

When  solving  problems,  a learner  combines  previously  learned  elements  of  knowledge,  rules,  techniques, 
skills,  and  concepts  to  provide  a solution  to  a novel  situation.  It  is  generally  accepted  that  mathematics  is  both 
process  and  product:  both  an  organized  body  of  knowledge  and  a creative  activity  in  which  the  learner  participates. 

It  might,  in  fact,  be  claimed  that  the  real  purpose  of  learning  rules,  techniques,  and  content  is  to  enable  the  learner  to 
do  mathematics,  indeed  to  solve  problems  (Orton,  1987).  Thus  problem  solving  can  be  considered  to  be  the  real 
essence  of  mathematics.  Gagne  (1985)  has  expressed  the  view  that  problem  solving  is  the  highest  form  of  learning. 
Having  solved  a problem,  one  has  learned.  One  might  only  have  learned  to  solve  that  problem,  but  it  is  more  likely 
that  one  has  learned  to  solve  a variety  of  similar  problems  and  perhaps  even  a variety  of  problems  possessing  some 
similar  characteristics.  Jonassen,  Howland,  Moore,  and  Marra  (2003)  also  point  out  that  solving  problems  are 
meaningful  kind  of  learning  activity  in  educational  settings. 

Problem  solving  activities  introduce  difficulties  for  management  by  teachers.  For  example:  choosing  and 
sequencing  problem  solving  tasks,  determining  the  degree  and  type  of  assistance  to  be  given  to  students,  maintaining 
motivation,  and  knowing  how  to  consolidate  understanding  through  reflection  and  follow-up  discussions,  demand 
continuous  decision  making  on  the  part  of  teachers.  Similarly,  learners  have  complicated  tasks  to  complete  during 
problem  solving,  especially  when  solving  word  problems.  Word  problems  are  set  in  specific  contexts  from  which 
students  have  to  develop  representations.  Learners  have  to  link  representations  to  appropriate  mathematical 
formulations,  and  to  apply  appropriate  techniques  in  order  to  produce  a solution.  Learners  have  to  monitor  and 
evaluate  this  problem  solving  process,  and  consider  the  implications  of  the  solution.  In  consequence,  giving 
assistance  and  managing  problem  solving  in  ways  which  create  effective  learning  and  the  development  of  cognitive 
skills  is  not  an  easy  task. 

Research  shows  that  a major  source  of  difficulty  experienced  by  children  in  the  problem  solving  process  is 
transforming  the  written  word  into  mathematical  operations  and  the  symbolization  of  these  operations.  Namely, 
children  are  required  to  disembed  the  information  from  the  problem  context,  select  the  relevant  values,  and  insert 
them  into  some  formula.  However,  children  are  not  very  successful  in  transferring  their  abilities  to  solve  problems  to 
subsequent  problems  (Jonassen  et  al.,  2003).  Orton  (1987)  indicated  that  the  most  common  difficulty  of  problem 
solving  is  failure  to  use  known  information.  He  also  noted  that  in  order  to  cope  with  this  difficulty  pupils  should  (1) 
write  the  problem  in  primitive  form  and  sketch  an  accurate  picture  of  the  setup  (where  applicable),  (2)  transform  the 
primitive  statements  to  simpler  language,  and  (3)  translate  verbal  problems  to  more  abstract  mathematical 
statement(s)  and  figures,  diagrams,  charts  and  other  similar  representations. 

Multiple  Representations 

Recent  approaches  to  mathematics  instruction  in  the  classroom  emphasize  mathematics  as  flexible, 
insightful  problem  solving  that  requires  understanding  that  mathematics  involves  pattern  seeking,  experimentation, 
hypothesis  testing,  and  active  seeking  of  solutions.  But  children’s  beliefs  about  the  nature  of  mathematics  contrast 
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with  this  emphasis.  For  example,  Baroody  (1987)  asserts  that  due  to  an  overemphasis  on  ‘the  right  answer’,  children 
commonly  believe  that  all  problems  must  have  a correct  answer,  that  there  is  only  one  correct  way  to  solve  a 
problem  and  that  inexact  answers  or  procedures  (such  as  estimates)  are  undesirable.  In  order  to  recognize  that 
multiple  solutions  and  different  representations  of  problems  are  possible,  children  need  to  have  higher  order  problem 
solving  skills.  Polya  (1962)  advocated  that  solvers  should  choose  multiple  representations  when  they  begin  to  solve 
a problem.  Jiang  and  Mcclintock  (2000)  also  suggested  that  encouraging  multiple  solutions  to  problem  solving  plays 
an  important  role  in  facilitating  students’  understanding  of  mathematical  concepts  and  their  grasp  of  methods  of 
mathematical  thinking.  In  this  way,  the  National  Council  of  Teachers  of  Mathematics  (NCTM,  2000)  states, 
“representations  should  be  treated  as  essential  elements  in  supporting  students’  understanding  of  mathematical 
concepts  and  relationships;  in  communicating  mathematical  approaches,  arguments,  and  understandings  to  one’s  self 
and  to  others;  in  recognizing  connections  among  related  mathematical  concepts;  and  in  applying  mathematics  to 
realistic  problem  situations  through  modeling”  (p.  67). 

Representations  are  mainly  divided  into  two  categories.  External  representations  are  the  knowledge  and 
structure  in  the  environment,  as  physical  symbols,  objects,  or  dimensions  and  as  external  rules,  constraints,  or 
relations  embedded  in  physical  configurations  (Zhang,  1997,  p.  180).  Internal  representations  are  retrieved  from 
memory  by  cognitive  processes.  External  and  internal  representations  are  particularly  beneficial  for  learning  when 
they  are  multiple.  In  most  cases,  learners  have  to  process  multiple  representations,  including  graphics,  symbols  and 
audio.  Classroom  teaching  has  traditionally  employed  multiple  external  representations  (MERs)  in  the  pursuit  of 
helping  students  learn.  Teachers  use  MERs  explicitly  in  order  to  make  abstract  situations  more  concrete. 

Kaput  (1992)  proposed  that  multiple  linked  representations  might  allow  learners  to  perceive  complex  ideas 
in  a new  way  and  to  apply  them  more  effectively.  By  providing  a rich  source  of  representations  of  a domain,  one  can 
supply  learners  with  opportunities  to  build  references  across  these  representations.  Such  knowledge  can  be  used  to 
expose  underlying  structure  in  the  domain  represented.  According  to  this  view,  mathematics  knowledge  can  be 
characterized  as  the  ability  to  construct  and  map  across  different  representations. 

Computer-Based  Multiple  Representations 

Computer  environments  have  been  gaining  great  importance  in  education.  Numerical  computation  tools  can 
be  used  by  problem  solvers  to  emphasize  planning  and  interpretation  of  arithmetic  operations.  The  existence  of 
computer  graphics  tools  can  be  used  to  help  students  understand  abstract  mathematical  concepts,  to  create  entirely 
new  graphic  oriented  representations  of  traditional  mathematical  topics,  or  to  provide  alternative  visual  methods  in 
mathematical  problem  solving.  As  Kaput  (1992)  states  for  the  case  of  mathematics  education,  this  entails  that 
routine  computations  can  be  off-loaded  to  a machine,  that  new  representational  mechanisms  only  available  on 
computers  (such  as  programs  as  representations)  become  available,  and  that  one  can  reify  abstract  concepts  by 
means  of  computer  simulations,  making  them  more  readily  accessible  for  reflection  and  dialogue. 

Fey  (1989)  asserts  that  the  use  of  numerical,  graphic  and  symbol  manipulation  is  a powerful  technique  for 
mathematics  teaching  and  learning.  He  identified  several  ways  in  which  computer-based  representations  of 
mathematical  ideas  are  unique  and  especially  promising  as  instructional  and  problem  solving.  First,  computer 
representations  of  mathematical  ideas  and  procedures  can  be  made  dynamic  in  ways  that  no  text  or  chalkboard 
diagram  can.  Second,  the  computer  makes  it  possible  to  offer  individual  students  an  environment  for  work  with 
representations  that  are  flexible,  but  at  the  same  time,  constrained  to  give  corrective  feedback  to  each  individual  user 
whenever  appropriate.  Third,  the  electronic  representation  plays  a role  in  helping  move  students  from  concrete 
thinking  about  an  idea  or  procedure  to  an  ultimately  more  powerful  abstract  symbolic  form.  Fourth,  the  versatility  of 
computer  graphics  has  made  it  possible  to  give  entirely  new  kinds  of  representations  for  mathematics-representation 
that  can  be  created  by  each  computer  user  to  suit  particular  purposes.  Finally,  the  machine  accuracy  of  computer 
generated  numerical,  graphic,  and  symbolic  representations  make  those  computer  representations  available  as 
powerful  new  tools  for  actually  solving  problems  (p.  255). 

Designing  the  Interactive  Learning 

The  use  of  multimedia  technology  has  offered  an  alternative  way  of  delivering  instruction.  The  old  text- 
based  approach  to  learning  is  being  superseded  by  an  approach,  which  includes  multisensory  representations 
(Jonassen  et  al.,  2003).  Interactive  multimedia  is  one  of  the  most  promising  technologies  of  the  time  and  has  the 
potential  to  revolutionize  the  way  we  work,  learn,  and  communicate  (Macromedia,  1992;  Staub  & Wertherbe,  1989). 
Interactive  multimedia  programs  take  the  idea  of  learning  and  doing  seriously.  With  interactive  multimedia 
programs,  the  learning  process  is  modified  by  the  actions  of  the  learners,  thus  changing  the  roles  of  both  the  learner 
and  the  teacher.  Interactive  multimedia  learning  is  also  a process,  rather  than  a technology,  that  places  new  learning 
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potential  into  the  hands  of  users  (Jonassen,  1999).  The  ideal  interactive  learning  environment  (ILE),  then,  is  one 
where  students  are  encouraged  to  undertake  such  activities  and  are  provided  with  feedback  as  they  do  so. 

Brooks  (1993)  stated  that,  with  all  the  additional  capabilities  of  the  growing  number  of  multimedia 
applications,  the  design  of  these  applications  has  become  a nightmare.  He  also  pointed  out  the  preponderance  of 
ugly  interfaces  containing  screens  full  of  multiple  fonts,  insignificant  boxes,  irrelevant  noises,  and  confusing  webs 
of  possible  interactivity  among  the  features  of  poorly  designed  multimedia  packages.  There  are  many  requirements 
that  must  be  checked  while  designing  an  interface  such  as  screen  design,  learner  control  and  navigation,  use  of 
feedback,  student  interactivity,  and  video  and  audio  elements  (Stemler,  1997).  So,  the  design  of  the  interface,  which 
considers  interactivity,  is  clearly  important  (Frye  et  al.,  1988).  Hence  a properly  designed  interface  should  make  the 
cognitive  process  transparent  and  externalized  so  as  to  support  evaluation,  reflection,  and  discussion  and  direct 
accessibility. 

The  following  principles  should  be  considered  during  the  design  of  ILEs  (Akpinar  & Hartley,  1996). 

The  ILE  should  provide  interactive  objects  and  operators,  which  are  visual  and  can  be  directly  manipulated 
by  pupils. 

The  ILE  system  should  provide  mechanisms  for  pupils  to  check  the  validity  of  their  methods,  and  thus 
receive  some  feedback  on  the  appropriateness  of  their  actions  in  relation  to  task. 

As  the  instruction  aims  to  support  links  between  the  concrete  and  symbolic  representation  of  word 
problems,  the  ILE  should  be  able  to  display  these  forms  so  that  the  equivalence  between  is  apparent.  The  system 
should  also  be  able  to  move  its  presentation  modes  to  the  symbolic  as  students  gain  in  competence. 

The  ILE  should  allow  experimentation  of  concepts  and  procedures  in  ways  that  relate  to  the  children’ s 
experiences.  In  brief  the  ILE  should  be  able  to  support  guided  discovery  as  well  as  directed  methods  of  instruction. 

The  ILE  should  allow  the  learning  to  be  conceptualized  and  procedural  in  its  approach,  and  be  capable  of 
adjusting  to  the  task  needs  of  teachers. 

Interactive  Learning  Environment:  LaborScale 

The  overall  aim  of  this  research  was  to  investigate  the  design  of  the  LaborScale  ILE  that  can  assist  problem 
solving  performance  and  understanding  specifically  mathematical  work  and  pool  problems.  Problem  solving 
requires  the  integration  and  utilization  of  multiple  knowledge  representations  e.g.  graphical,  symbolic,  and  audio. 
Depending  upon  these  factors,  the  design  of  the  computer  based  learning  environment  must  take  into  consideration 
students’  knowledge  so  that  it  can  accommodate  different  levels  of  competence  and  be  useful  for  varying  modes  of 
instruction  in  the  classroom  (Mayer  1985).  The  proposed  design  to  realize  these  aims  is  LaborScale  that  provides  a 
constructive  environment  based  on  direct  manipulation,  user- system  interactions  and  available  interface  design. 

The  ILE  should  have  all  the  features  and  components  to  develop  children’s  word  problem  solving  skills. 
Hence,  it  should  be  designed  with  an  object-oriented  and  direct-manipulation  approach.  In  order  to  reach  this 
objective,  the  instructional  software  (LaborScale)  was  developed  and  implemented  by  the  researchers  using  an 
authoring  tool,  Asymetrix  Toolbook  II  5.0  and  other  related  multimedia  programs  (Macromedia  Flash  5.0,  3D 
Studio  Max  and  Photoshop  5.0)  to  support  Toolbook  Application  with  videos,  animations,  audios  and  pictures.  The 
user-friendliness  of  Toolbook  interface  and  its  accompanied  object-oriented  scripting  language,  Openscript,  used  to 
specify  the  functionality  were  well  suited  to  the  development  and  implementation  of  the  prototype.  Toolbook 
software  is  a development  environment  that  provides  tools  to  draw  objects  that  can  be  made  interactive  using  the 
Openscript  programming  language.  Also  the  development  can  be  carried  out  incrementally.  Further,  Toolbook  is 
event-driven  i.e.  an  application  can  respond  to  events  such  as  mouse  clicking  whenever  they  occur.  This  is  suitable 
for  the  ILE  interface  that  is  based  on  a direct-manipulation  approach  in  addition  to  its  ability  to  produce  quality 
visual  animation. 

The  user-interface  of  LaborScale  has  two-page  design  consisting  of  multiple  viewers  in  which  each  viewer 
has  a background  and  a foreground  containing  objects  such  as  fields,  buttons,  graphics  and  text.  The  user  interface 
has  two  units:  a curriculum-manager  unit  and  a student- working  unit. 

Curriculum  Manager  Unit 

Curriculum  Manager  Unit  (CMU)  is  one  of  the  main  windows  of  the  LaborScale  (Figure  1).  This  is  the 
place  where  teachers  set  problems  and  customize  environments  for  students.  Each  problem  specification  will  need  to 
provide  context  information,  and  the  concepts  based  on  the  activity  sets.  Hence,  the  purpose  of  the  problem 
specification  is  to  provide  contexts  familiar  to  the  students. 
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The  CMU  has  been  designed  to  manage  the  specification  of  activity  sets  that  contains  the  problem  content, 
specification  of  problems,  and  types  of  representation.  In  order  to  form  a new  activity  set  for  any  student,  teachers 
can  use  two  methods.  One  has  two  steps,  which  are  pre-storing  problems  and  their  answers,  and  specification  of  the 
problems  depending  on  the  level  of  the  students.  The  other  is  only  specification  of  problems,  which  are  saved  to  the 
system  previously.  In  brief,  teachers  can  manage  the  following  tasks  by  using  CMU: 

• Forming  problem  sets  including  simple  and  advanced  level  problems  of  two  types,  work  and  pool,  by 
saving  problems  to  the  systems, 

• Setting  audio  environment  of  student- working  unit, 

• Preparing  an  activity  set  with  respect  to  the  students’  level, 

• Looking  at  the  performances  of  the  students  who  finish  their  activity  sets, 

• Viewing  the  activity  set  in  the  student- working  unit. 

Figure  1 . A screen  of  curriculum  manager  unit 


Student  Working  Unit 

The  student  working  window  (Figure  2),  the  learner  mode  of  the  LaborScale,  is  the  second  main  window  of 
the  LaborScale.  The  Curriculum-Manager  Unit  passes  the  sequence  of  problems  to  the  ILE  controller  that  is  to 
manage  the  interactions  with  students  and  to  keep  records  of  their  progress.  Hence  a principal  consideration  in  the 
design  of  this  unit  was  the  user-system  interface  in  which  these  interactions  take  place.  LaborScale  is  based  on  a 
high  degree  of  graphical  and  symbolic  object  manipulation,  and  with  the  interface  users  are  able  to  directly 
manipulate  the  LaborScale  objects,  for  example  by  giving  the  values  symbolically,  dragging  and  dropping  of  picture 
of  these  values,  and  combining  the  representations  of  them  to  reach  a solution.  To  outline,  students  can  manage  the 
following  tasks  in  this  unit: 

• Displaying  ratios  they  entered  in  the  problems, 

• Dragging  and  dropping  the  displayed  objects  and  displaying  a vertical  scale  as  a result  of  this, 

• Reaching  right  answers  of  the  problems  by  analyzing  a horizontal  scale  depending  on  the  vertical  scale, 

• Setting  audio  environment, 

• Transition  to  other  problems. 

Figure  2.  A screen  of  student  working  unit 
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Evaluation  studies 

The  validation  of  LaborScale  was  carried  out  during  last  week  of  May  and  first  week  of  June  2001.  The 
method  of  the  research  was  pretest  and  posttest  group  design.  The  sample  of  the  study  was  selected  from  seventh 
grade  students  of  two  different  schools  (Public  school,  school  A and  private  school,  school  B)  that  have  a computer 
laboratory.  The  subjects  were  selected  by  using  clustering  sample  technique,  namely,  one  class  was  selected  from 
each  school.  The  validation  experiment  was  performed  as  a pretest  of  the  work  and  pool  problems.  The  pre-test  was 
administered  to  80  students  (59  from  public  school  and  21  from  private  school).  Their  average  age  was  14  and  40  of 
them  were  girls  and  40  of  them  were  boys.  For  the  application  and  post- test,  convenient  students  were  chosen  from 
each  class  by  considering  their  teacher’s  opinions  and  number  of  computers  in  the  computer  laboratories  of  both 
schools.  Students  were  chosen  according  to  their  achievements  for  each  mode  of  the  pre-test,  namely,  numerical 
solutions,  symbolic  and  graphic  representations  of  solutions.  The  students’  actions  in  LaborScale  were  recorded  by 
the  system  and  notes  were  taken  by  the  researcher  as  well  as  the  pre  and  posttest  differences  in  performance. 

The  instructor  organized  software  for  27  students  to  run  in  their  laboratories.  The  reason  for  choosing  only 
27  students  was  that  there  were  28  computers  in  the  computer  laboratory  of  School  A,  but  17  of  them  were  available 
to  run  the  software.  There  were  13  computers  in  the  computer  laboratory  of  School  B,  but  10  of  them  were  available 
to  run  the  software.  Five  problems  randomly  chosen  from  nine  problems  were  assigned  to  these  students  for  both 
work  and  pool  problems  respectively  by  regarding  the  pre-test  scores  of  the  students.  Pool  problems  were  given  to 
them  at  the  second  week  of  the  application  to  allow  students  to  manage  problems  at  ease  and  meaningfully.  For  low- 
achievers,  three  of  the  problems  were  simple  and  two  of  them  were  advanced.  For  intermediate  students  and  high 
achievers,  two  of  the  problems  were  simple  and  three  of  them  were  advanced. 

Before  the  application  of  the  software  in  both  schools,  the  researcher  provided  students  with  an  orientation 
session  at  which  the  students  received  explanations  and  were  experienced  on  how  they  will  use  the  program  by 
presenting  worked  examples  for  solving  different  type  of  problems.  This  session  lasted  about  20  minutes.  Since  the 
students  in  both  schools  had  a regular  computer  course,  they  had  no  difficulty  in  controlling  and  manipulating  the 
environment  during  the  instruction. 

After  an  orientation  session,  all  groups  received  computer-assisted  treatment  for  two  hours  without  any 
break  for  two  weeks  respectively.  During  the  instruction,  students  were  left  alone  and  they  only  interacted  with 
computers.  They  solved  their  own  problems  about  the  program  by  themselves  except  system  problems  by  using  the 
help  and  information  modes  of  the  software.  Therefore,  the  researcher  behaved  like  an  observer  in  the  application. 
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The  time  that  the  students  completed  their  activity  sets  changed  between  30  and  65  minutes  for  work  problems, 
however,  the  instruction  on  pool  problems  lasted  between  20  and  45  minutes. 

At  the  end  of  the  instruction  the  performance  test  (PT)  was  conducted  to  all  subjects  as  a post- test.  After  all, 
the  questionnaire  about  the  software  evaluation  was  applied  to  the  teachers  involved  in  the  study  to  obtain  their 
criticisms. 

In  order  to  analyze  the  differences  between  the  pre  and  post  tests  mean  scores  of  the  whole  group  and 
School  A obtained  from  each  mode  of  PT,  paired  sampled  t-test  was  used,  and  Wilcoxon  test  was  used  for  school  B 
since  the  numbers  of  subjects  in  the  groups  were  too  small. 

The  pre  and  posttests  results  showed  significant  improvements  in  students’  performances  for  each  mode 
that  pointed  to  the  benefits  of  LaborScale  ILE  (See  Table  1,  Table  2,  and  Table  3).  When  the  schools  were  analyzed 
separately,  there  was  a significant  increase  in  all  modes  of  the  tests  for  each  school.  Looking  at  the  posttest  results  of 
each  mode,  significant  improvements  were  also  observed  (for  more  details  see  Adiguzel,  2001). 


Table  1 Differences  between  pre-post  test  scores  of  numerical  solutions  to  PT 


Groups 

N 

Pre-test 
Num.  Sol. 
Mean 

Pre-test 
Num.  Sol. 
Std.  Dev. 

Post-test 
Num.  Sol. 
Mean 

Post-test 
Num.  Sol. 
Std.  Dev. 

t 

Sig. 

(2-tailed) 

Total 

27 

22.85 

27.28 

85.78 

21.70 

10.997* 

.000 

School  A 

17 

18.47 

25.36 

88.35 

24.43 

9.036* 

.000 

z 

School  B 

10 

30.30 

30.16 

81.40 

16.29 

2.805* 

.005 

*p  < .05. 

Conclusion 

Studies  formed  a base  for  using  computers  featuring  multiple  linked  representations  to  assist  students  with 
the  transition  from  concrete  experiences  to  abstract  mathematical  ideas,  with  the  practice  of  skills,  and  with  the 
process  of  problem  solving,  like  in  the  LaborScale  ILE,  namely,  beginning  with  the  concrete  representations  and 
reaching  the  symbolic  representations  by  using  visual  components  supported  by  audio  developed  seventh  grade 
students’  performance  on  work  and  pool  problems. 
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Table  2 Differences  between  pre-post  test  scores  of  symbolic  mode  of  PT 


Groups 

N 

Pre-test 

Symbolic 

Mean 

Pre-test 
Symbolic 
Std.  Dev. 

Post-test 

Symbolic 

Mean 

Post-test 
Symbolic 
Std.  Dev. 

t 

Sig. 

(2-tailed) 

Total 

27 

2.74 

8.86 

92.37 

11.27 

36.443* 

.000 

School  A 

17 

2.47 

9.68 

96.59 

5.49 

37.343* 

.000 

z 

School  B 

10 

3.20 

7.73 

85.20 

14.92 

2.807* 

.005 

*p  < .05. 

Table  3 Differences  between  pre-post  test  scores  of  graphic  mode  of  PT 


Groups 

N 

Pre-test 

Graphic 

Mean 

Pre-test 
Graphic 
Std.  Dev. 

Post-test 

Graphic 

Mean 

Post-test 
Graphic 
Std.  Dev. 

t 

Sig. 

(2-tailed) 

Total 

27 

18.67 

18.62 

82.26 

31.25 

10.380* 

.000 

School  A 

17 

21.65 

14.71 

95.00 

10.90 

16.858* 

.000 

z 

School  B 

10 

13.60 

23.92 

60.60 

42.21 

2.499* 

.012 

*p  < .05. 

In  the  post- test,  students  had  different  graphical  representations  of  solutions.  The  results  showed  that  they 
were  affected  from  the  visual  components  of  the  LaborScale  ILE,  namely,  some  of  them  drew  a box,  meant  whole 
work,  for  each  worker  and  they  indicated  the  results  by  the  graph  corresponding  these  related  graphs.  Some  of  them 
drew  a vertical  scale  whose  pointer’ s location  showed  the  values  of  work  done  in  a day  for  each  worker  and  the 
result.  The  rest  of  the  students  drew  a horizontal  scale,  similar  to  the  number  line,  whose  pointer’ s location  also 
represented  the  values  of  work  done  in  a day  for  each  worker  and  the  result.  This  proves  that  if  the  students  are 
given  more  visual  representations,  they  will  use  and  connect  them  to  the  symbolic  representation  of  these  and  they 
will  grasp  the  meaning  of  the  word  problem  solving  by  concretizing  them. 

The  increase  in  the  symbolic  mode  was  more  than  the  other  modes.  This  proves  that  multiple  linked 
representations  relating  the  symbolic  representation  to  graphic  representation  allow  learners  to  perceive  complex 
ideas  in  a new  way  and  to  apply  them  more  effectively.  However,  the  increase  in  the  graphical  mode  was  significant 
but  less  than  the  other  modes.  The  reason  for  this  may  be  that  since  text-based  books  were  dominant  in  the 
curriculum,  children  have  not  developed  to  present  problems  graphically.  However,  the  graphical  representations 
need  to  be  well  constructed  and  be  capable  of  representing  the  information  in  a problem  to  enable  the  processing 
capabilities  of  the  human  visual  system  to  be  exploited,  so  that  perceptual  features  and  judgments  can  be  developed 
and  related  to  a more  abstract  symbolic  understanding  (Cox  & Brna,  1995).  Also,  graphical  representations  are 
effective  problem  solving  and  learning  tools  because  they  reduce  the  space  of  applicable  operations  and  they  are 
more  specific  than  the  other  representations.  Relating  with  the  theory,  good  performance  on  finding  and  presenting  a 
graphic  representation  of  a solution  raised  the  performance  on  grasping  symbolic  representation  of  the  solution  in 
the  study. 

A similar  result  to  the  findings  of  this  research  was  the  outcome  of  ANIMATE  software  (Nathan,  1991). 
However,  the  potentials  and  facilities  of  ANIMATE  differ  from  the  LaborScale.  Though  Nathan’s  ANIMATE  can 
not  generalize  the  solution  method  into  an  algebraic  formula,  LaborScale  can  help  students  to  build  up  algebraic 
formula  that  may  be  generalized  and  employed  in  a wide  variety  of  problems 

During  the  application,  the  time  at  which  students  finished  each  activity  set  was  recorded  by  the  researcher. 
According  to  the  results,  students  spent  longer  time  period  on  the  first  question  of  the  activity  set  of  the  work 
problems  than  the  other  questions  of  the  activity  set  since  they  were  adapted  to  the  system  in  the  first  question.  After 
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they  gained  an  experience  on  the  work  problems,  they  acted  carefully  and  swiftly  on  the  pool  problems  since  the 
application  of  the  pool  problems  was  applied  to  them  one  week  later. 

Performance  recording  unit  of  the  program  stored  two  main  functions  of  the  students:  three  trials  on  the 
values  that  are  work  done  in  a day  for  each  worker  and  the  answer  of  the  problem  for  each  trial.  According  to  the 
records  obtained  from  this  unit,  all  students  had  tried  at  least  three  times  on  the  advanced  level  problems  and 
problems  requiring  complex  calculation  to  reach  a right  answer.  The  reason  for  this  was  the  greatness  of  the  least 
common  multiple  related  with  the  value  of  denominator  of  the  work  done  in  one  day  and  that  because  of  the  big  unit 
differences  in  the  scale  students  could  not  grasp  net  measurement.  However,  they  usually  solved  these  problems 
after  they  tried  at  least  three  times.  The  main  thing  in  this  part  is  requiring  the  students  to  grasp  the  values  from  the 
problem.  As  a result,  students’  problem  solving  skills  significantly  improved  by  the  help  of  instructional  software, 
LaborScale. 

The  rationale  of  the  development  of  the  LaborScale  was  to  base  students’  problem  solving  on  multiple 
representations.  This  should  aid  understanding,  conform  to  problem  solving  as  an  investigatory  and  creative  activity. 
While  LaborScale  was  successful  in  fulfilling  many  of  these  claims  in  its  design  and  conception,  there  are  further 
requirements  to  complement  or  supplement  its  current  facilities  and  strengths. 

In  conclusion,  the  LaborScale  software  and  validation  studies  have  given  some  support  to  the  design 
principles  of  ILEs  in  which  the  aim  is  to  produce  user-system  interfaces  that  release  students’  knowledge  and 
stimulate  active  and  investigatory  methods  of  learning.  The  suggested  further  work  could  re-illuminate  design 
principles  for  multi-representational  interfaces.  And,  since  the  research  was  concluded  successfully  and  significant 
results  were  obtained,  hopefully  it  should  be  adapted  for  other  domains  of  mathematics. 
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Investigating  the  Relationships  Among  Instructional  Strategies  and  Learning 

Styles  in  Online  Environments 
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Tiffany  A.  Koszalka 
Syracuse  University 

Abstract 

Researchers  investigated  differences  in  learner  preferences  for  different  types  of  instructional  strategies 
and  learning  styles  in  online  environments.  Results  suggested  that  matches  between  students  ’ learning  styles  and 
instructional  strategies  did  not  affect  their  perception  of  their  own  learning  outcomes,  level  of  effort  and 
involvement,  and  level  of  interactions  in  the  course.  Data  also  indicated  that  no  single  instructional  strategy,  among 
three  instructional  strategies  tested,  emerged  as  superior  for  high  and  low  field  dependent  online  students. 

Introduction 

The  Internet  has  taken  center  stage  today  as  a preferred  medium  for  the  delivery  of  distance  education. 
Many  universities  offer  online  courses  that  respond  to  the  diverse  distance  and  time  needs  of  today’s  learners.  These 
universities  provide  course  instructors  with  online  tools  to  manage  course  participation  and  facilitate  learning. 
Instructors  can  continuously  monitor  student  progress,  provide  learners  with  time  to  reflect  on  content  and  feedback 
before  participating,  prompt  active  participation  with  content  and  peers,  and  offers  instructional  modules  that  are 
designed  to  appeal  to  a variety  of  learning  styles  and  preferences  (Hamilton-Pennell,  2002). 

Learning  style  can  be  thought  of  as  the  combination  of  the  learners’  motivation,  task  engagement,  and 
information-processing  habits  (Aragon,  Johnson,  & Shaik,  2002).  Each  learner  can  have  different  preferences  as  to 
how  s/he  receives,  processes,  and  recalls  information  during  instruction.  Many  researchers  however,  have  not 
controlled  for  students’  characteristics  in  their  analyses  of  students’  satisfaction  of  online  instruction  (Thurmond, 
Wambach,  & Connors,  2002).  Understanding  the  relationships  among  learning  styles  and  instructional  preferences 
holds  great  promise  for  enhancing  educational  practice  (Claxton  & Murrell,  1987). 

The  primary  purpose  of  this  exploratory  pilot  study  was  to  investigate  the  relationships  among  learning 
styles,  defined  as  high  and  low  field  dependence,  and  preferences  for,  and  evaluation  of,  instructional  strategies  used 
in  an  online  course.  Field  dependence  describes  the  degree  to  which  a learner’s  perception  or  comprehension  of 
information  is  affected  by  the  surrounding  contextual  field  (Jonassen  & Grabowski,  1993).  Learning  styles  are 
useful  because  they  provide  information  about  individual  differences  from  a cognitive  and  information-processing 
standpoint  (Smith  & Ragan,  1999).  Field  dependent  individuals  are  more  likely  to  succeed  at  learning  tasks  that 
engage  them  in: 

• Group  oriented  and  collaborative  work  situations 

• Situations  where  individuals  have  to  follow  standardized  pattern  of  performance 

• Tests  requiring  individuals  to  recall  information  in  the  form  or  structure  that  it  was  presented  (Jonassen  & 

Grabowski,  1993). 

High-field  dependent  individuals  have  more  difficulty  locating  the  information  they  are  looking  for  than 
low  field  dependent  individuals.  Low  field  dependent  individuals  are  more  likely  to  excel  at  learning  tasks  involving 
identification  of  important  aspects  of  information  from  a poorly  organized  body  of  information.  High  field 
dependent  individuals  tend  to  accept  the  information  without  reorganizing  it  from  the  way  it  was  presented  to  them 
so  low  field  dependent  individuals  are  likely  to  reorganize  information  to  fit  their  own  perceptions.  Muir  (2001) 
recommends  teaching  methods  that  match  instructional  strategies  to  field  dependence-independence  style. 

Instructional  strategies  represent  a set  of  decision  that  result  in  plan,  method,  or  series  of  activities  aimed  at 
obtaining  a specific  goal  (Jonassen,  Grabinger,  & Harris,  1990).  Instructional  strategies  are  the  activities  used  to 
engage  learners  in  the  learning  process.  Many  types  of  instructional  strategies  are  used  to  engage  learner  in  different 
ways  such  as  reading,  collecting,  thinking,  etc.  Expository  strategies  may  include  providing  learners  with  lecture 
notes.  Explanations  are  often  kept  simple  and  direct.  Students  usually  use  lecture  notes  to  complete  learning 
activities  or  respond  to  posed  questions.  Collaborative  and  group  work  instructional  strategies  require  individuals, 
often  at  various  levels,  to  work  together  to  achieve  a common  goal.  Individuals  are  prompted  to  analyze,  synthesize, 
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and  evaluate  their  ideas  collaboratively.  Inquisitive  (discovery  learning)  instructional  strategies  require  individuals 
to  formulate  investigative  questions,  obtain  factual  information,  and  build  knowledge,  which  reflects  their  answer  to 
the  original  question.  Students  develop  several  questions,  which  eventually  lead  them  to  answer  the  original 
question,  use  extensive  resources  to  gather  data,  and  answer  the  original  question. 

The  characteristics  of  high  field  dependent  individuals  appear  to  match  with  expository  (presentation),  and 
collaborative  (group  work)  types  of  strategies  because  these  types  of  instructional  strategies  require  learners  to 
complete  learning  activities  that  are  usually  kept  simple,  and  sometimes  require  learners  to  work  together.  The 
characteristics  of  low  field  dependent  individuals  suggest  a match  with  inquisitive  type  of  strategies  because  low 
field  dependent  individuals  prefer  generating  their  own  hypothesis  and  testing  their  hypothesis.  Table  1 illustrates 
the  suggested  match  and  mismatch  of  learning  style  and  instructional  strategy  for  this  study. 


Table  1.  Match  and  Mismatch  of  Learning  Style  and  Instructional  Strategy 


Expository 

Collaborative 

Discovery 

High  Field  Dependent 

Match 

Match 

Mismatch 

Low  Field  Dependent 

Mismatch 

Mismatch 

Match 

Abraham  (1985)  found  that  matching  instructional  styles  to  students’  field-dependent  or  independent  style 
improved  students’  performance  in  the  course.  In  the  study,  researchers  used  two  computer-assisted  instruction 
lessons,  one  rule  oriented,  and  the  other  deemphasizing  rules,  to  test  whether  a teaching  approach  that  did  not 
emphasize  rules  would  be  of  greater  benefit  to  field-dependent  students  in  an  English  as  a second  language  class. 
The  results  of  the  study  showed  that  field-independent  students  performed  better  with  rule  oriented  approach 
whereas  field-dependent  students  performed  better  with  the  approach  deemphasizing  rules.  There  has  also  been 
research  that  was  contradictory  to  these  results.  Macneil  (1980)  found  that  learning  did  not  increase  when  students 
categorized  as  field  dependent  and  field  independent  receive  instruction  oriented  to  their  style.  In  the  study, 
researchers  used  discovery  and  expository  approaches  to  test  whether  randomly  assigned  field  dependent  students 
learn  more  from  the  discovery  approach  and  field  independent  students  learn  more  from  expository  approach. 
Results  of  the  study  revealed  that  achievement  of  field  dependent  and  field  independent  students  did  not  vary  as  a 
function  of  style.  The  question  remains  can  matching  learning  styles  and  instructional  strategies  in  distance 
education  better  support  student  learning.  This  study  was  designed  to  address  the  following  research  questions: 

Is  there  a difference  in  perceived  learning  outcomes  for  students  whose  learning  style  matches  with  the 
instructional  strategy? 

Is  there  a difference  in  students’  effort  and  involvement  for  students  whose  learning  style  matches  with  the 
instructional  strategy? 

Is  there  a difference  in  students’  perceived  level  of  interaction  for  students  whose  learning  style  matches  with 
the  instructional  strategy? 

Is  there  a difference  in  perceived  learning  outcomes  for  low  field  dependent  learners  in  match  and  mismatch 
instructional  strategy  situations? 

Is  there  a difference  in  perceived  learning  outcomes  for  high  field  dependent  learners  in  match  and  mismatch 
instructional  strategy  situations? 


Method 

Instructional  Context 

The  pilot  study  was  conducted  at  a private  university  located  in  the  northeastern  United  States  with 
graduate  students  enrolled  in  an  online  graduate  course  entitled  Design  and  Management  of  Distance  Education. 

This  investigation  focused  on  determining  if  students  who  were  classified  as  low  or  high  field  dependent  perceived 
different  types  of  instructional  strategies  differently  in  an  online  instructional  environment.  Specifically,  students 
would  be  queried  about  their  perceptions  of  learning  outcomes,  their  effort  and  involvement  in  the  activities,  and 
their  level  of  interaction  during  the  course. 

The  Design  and  Management  of  Distance  Education  course  consisted  of  three  modules.  Each  module  was 
delivered  online  using  a different  instructional  strategy  including,  expository  (presentation),  collaborative  (group 
work),  and  inquisitive  (discovery  learning).  All  three  units  were  experiential  and  generative  in  nature,  requiring 
learners  to  interact  in  different  ways  with  the  content  to  facilitate  learning.  On  average,  each  unit  was  completed 
over  a four- week  period. 

Expository  type  of  instructional  strategy  was  utilized  primarily  to  present  module  one  content.  Each  student 
read  the  assigned  chapters  in  the  course  text,  specified  web  pages,  and  power  point  slides  regarding  the  growth  and 
development  of  the  field  of  distance  education.  Students  were  then  required  to  participate  asynchronous  discussions 
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responding  to  initial  question  posted  by  course  professor  and  at  least  two  other  postings  from  their  peers  supporting 
their  responses  with  references  from  readings.  Finally  students  were  required  to  write  a reflection  journal  and 
complete  content  quizzes. 

Module  two  was  presented  using  collaborative  group  work.  Four  teams  of  3 to  4 students  were  established. 
A case  scenario  was  presented  and  each  team  was  asked  to  design  a prototype  distance  education  course  based  on 
specified  criteria.  A private  discussion  forum  and  workspace  was  made  available  to  each  team  to  support  their 
collaboration  while  completing  the  module.  Throughout  the  module,  each  team  was  expected  to  submit  status 
reports,  and  a final  instructional  design  report.  Quality  of  the  deliverables  and  level  of  participation  were  used  as 
evaluation  criteria. 

Inquisitive  (discovery  learning)  types  of  strategies  were  used  to  present  module  three.  Students  were 
prompted  to  explore  methods,  media,  and  materials  in  distance  education,  to  identify  most  important  points  of  their 
implementation,  and  to  prepare  a mini  presentation  describing  benefits  and  challenges  of  each.  In  addition  to  the 
course  text,  and  additional  web  links,  students  were  expected  to  utilize  other  resources  to  prepare  the  mini 
presentation.  Then,  students  were  expected  to  participate  in  a bulletin  board  discussion,  write  a reflection  journal 
describing  the  at  least  five  web  sources  helping  them  to  better  understand  on  hot  topic  in  distance  education  related 
to  methods,  media,  or  materials.  For  example,  if  a student  was  curious  about  copyright  s/he  would  explore  the  topic 
and  report  findings  back  to  class.  Ultimately,  students  were  prompted  to  respond  to  inquiries  into,  and  learn  about 
distance  education  by  investigating  a variety  of  distance  education  areas  of  their  own  choice,  and  share  their  findings 
with  the  class. 

Subjects 

The  subjects  included  twelve  graduate  students  registered  for  this  course.  Sixty-six  percent  of  the  students 
were  doctoral  students  and  others  were  master  degree  students.  Four  students  reported  their  technical  skill  as 
advanced.  The  other  eight  studied  described  their  technical  skills  as  intermediate.  Sixty-six  percent  of  the  students 
had  taken  at  least  one  online  course  before  enrolling  in  this  course.  The  results  of  the  Psychological  Differentiation 
Inventory  showed  that  25%  of  the  students  were  high  field  dependent  and  others  were  low  field  dependent  students. 

Instruments 

In  order  to  conduct  this  research  a valid  and  reliable  measure  of  learning  style  had  to  be  secured  that  could 
be  implemented  online.  One  such  measure  used  for  decades  to  study  learning  styles  is  the  Group  Embedded  Figures 
Test  (GEFT)  (Witkin  et  al.,  1971).  The  GEFT  is  used  for  measuring  field  dependence  and  independence.  However, 
the  use  of  this  instrument  is  problematic  for  online  environments  because  of  the  requirement  to  time  participant 
responses  and  because  participants  have  to  draw  responses  in  a given  booklet.  Given  that  distributed  nature  of 
students,  the  reliability  of  each  participant  completing  the  instrument  per  instruction  is  questionable.  Therefore,  the 
investigator  searched  for  a version  of  the  instrument  that  could  be  implemented  online.  The  Psychological 
Differentiation  Inventory  (PDI),  a questionnaire  measure  of  field  dependence  was  reconstructed  as  an  online 
questionnaire  for  this  study  and  used  to  measure  high  field  dependence  and  low  field  dependence  of  participating 
learners.  The  PDI  has  good  test-retest  reliability  (.69)  and  correlates  (r  = 0.  46  - 0.76)  with  Embedded  Figure  Test 
which  is  frequently  used  as  a single  measure  of  field  dependence  (Evans,  1969). 

In  this  research  the  evaluation  system  used  to  assess  students’  achievements  in  each  module  included  three 
components.  These  components  were  (1)  self-assessment  of  outcome,  (2)  individual  effort  and  involvement,  and  (3) 
interaction  and  feedback  between  and  among  the  instructor  and  students  (Robles  & Braathen,  2002).  The  modified 
version  of  Student  Instructional  Report  II  developed  by  John  A.  Centra  in  1998  was  used  with  permission  to  assess 
components  1 and  2.  This  instrument  contains  five  items  for  assessing  perceived  unit  outcome  of  students,  and  three 
items  for  assessing  student  effort  and  involvement.  Returns  indicated  the  student’ s perception  of  the  effectiveness  of 
each  aspect  of  a unit  to  the  same  aspects  in  other  units  using  a five-point  scale.  A rubric  developed  by  Roblyer  & 
Wiencke  in  2003  was  used  to  assess  the  level  of  interactivity  in  each  module  by  having  students  evaluate  elements 
of  interactions  including  social  goals,  instructional  goals,  types  and  uses  of  technology,  and  impact  of  interactivity  - 
changes  in  learner  behaviors. 

Procedure 

The  Design  and  Management  of  Distance  Education  course  consisted  of  three  modules.  Each  module  had 
to  be  completed  in  order,  and  in  a given  time  frame  by  all  students.  Data  were  collected  after  each  unit  was 
completed.  The  online  unit  evaluation  form  at  the  end  of  each  unit  measured  learner  satisfaction  and  involvement 
with  the  instruction  specifically  through  (1)  perceived  unit  outcomes,  (2)  student  perception  of  effort  and 
involvement  in  the  unit,  and  (3)  student  perception  of  interaction  and  feedback  levels  between  and  among  the 
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instructor  and  students  during  the  unit  (Roblyer  & Wiencke,  2003;  Centra,  1998).  A java  script  was  written  for  the 
online  unit  evaluation  form  to  ensure  that  students  answered  all  questions  before  submitting  it.  Using  java  script 
eliminated  the  risk  of  missing  question  response.  Upon  completing  the  online  unit  evaluation  form,  the  data  were 
automatically  emailed  to  the  researchers. 

Students  also  had  to  complete  the  online  questionnaire  version  of  the  Psychological  Differentiation 
Inventory  to  measure  their  level  of  field  dependence.  A java  script  was  also  written  for  the  online  questionnaire 
version  of  the  Psychological  Differentiation  Inventory  to  ensure  that  students  answered  all  questions  on  the 
inventory.  Researchers  also  received  the  results  of  the  Psychological  Differentiation  Inventory  through  email. 

Analysis 

All  data  were  ported  into  a statistical  analysis  package  (Stata  version  8.0)  for  later  analysis.  One  way 
analysis  of  variance  was  used  to  test  the  hypotheses  that  there  were  differences  in  students  perceived  learning 
outcomes,  students  effort  and  involvement,  and  students’  perceived  level  of  interaction  when  students  learning  style 
matches  with  the  instructional  strategy,  and  to  test  whether  one  instructional  strategy  emerges  with  higher  perceived 
learning  outcomes  for  online  students  who  are  categorized  as  high  field  dependent  and  low  field  dependent.  All 
statistical  analysis  reported  in  this  research  were  conducted  with  a significant  level  of  .05. 

Results 

Learning  style 

The  results  of  the  online  questionnaire  version  the  Psychological  Differentiation  Inventory  revealed  that 
nine  students  were  low  field  dependent  and  three  students  were  high  field  dependents.  The  mean  score  for  students 
categorized  as  low  field  dependent  was  19.55  (S.D.  = 3.53)  while  the  mean  score  for  students  categorized  as  high 
field  dependent  was  26.33  (S.D.  = 0.57)  (see  Table  2). 


Table  2.  Means  and  Standard  Deviations  for  Students  Categorized  as  Low  Field  Dependent  and  High  Field 
Dependent 


Categories 

N 

Mean 

Standard 

Deviation 

Min 

Max 

Low  Field 
Dependent 

9 

19.55 

3.53 

14 

23 

High  Field 
Dependent 

3 

26.33 

0.57 

26 

27 

Matching  Learning  Style  with  Instructional  Strategy 

The  first  hypothesis  stated  that  there  would  be  no  significant  difference  in  the  perceived  learning  outcomes 
of  students  whose  learning  style  matched  the  instructional  strategy.  The  results  of  the  one-way  analysis  of  variance 
supported  this  null  hypothesis,  F (2,18)  = 0.11,  p = 0.89  (see  Table  3).  No  significant  difference  was  found  in  the 
perceived  learning  outcomes  of  students  whose  learning  style  matched  the  instructional  strategy.  Both  low  and  high 
field  dependent  students  perceived  learning  outcomes  in  the  three  instructional  strategies  the  same.  Table  4 shows 
the  descriptive  statistics  for  perceived  learning  outcomes  of  students  whose  learning  style  matched  the  instructional 
strategy. 

Table  3.  Results  of  One-way  Analysis  of  Variance  for  Perceived  Learning  Outcomes  of  Students  whose  Learning 
Style  Matched  the  Instructional  Strategy  Used  to  Present  the  Online  Course  Module 


Source 

Sum  of  Squares 

D.F. 

Mean  Squares 

F ratio 

F Prob. 

Between  groups 

.18031733 

2 

.090158665 

0.11 

0.8947 

Within  groups 

14.4977771 

18 

.805432064 

Total 

14.6780945 

20 

.733904724 

Table  4.  The  Descriptive  Statistics  for  Perceived  Learning  Outcomes  of  Students  whose  Learning  Style  Matched  the 
Instructional  Strategy 


Matched  Group 

Instructional  Strategy 

Mean 

N 

S.D. 

Min 

Max 

Low  Field  Dependent 

Expository 

3.71 

9 

0.85 

2 

5 

Low  Field  Dependent 

Collaborative 

3.55 

9 

0.88 

2 

5 

High  Field  Dependent 

Discovery 

3.46 

3 

1.1 

2.4 

4.6 
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The  second  hypothesis  stated  that  there  would  be  no  significant  difference  in  the  effort  and  involvement  of 
students  whose  learning  style  matched  the  instructional  strategy  used  to  present  the  online  course  module.  The 
results  of  the  one  way  analysis  of  variance  supported  this  null  hypothesis,  F(2,18)  = 1.02,  p = 0.37  (see  Table  5).  No 
significant  difference  was  found  in  the  effort  and  involvement  of  students  whose  learning  style  matched  the 
instructional  strategy  used  to  present  the  online  course  module.  When  low  and  high  field  dependent  students’ 
learning  styles  matched  three  types  of  instructional  strategies  used  in  the  study,  low  and  high  field  dependent 
students  reported  they  put  equal  effort  and  involvement  to  instructional  activities.  Table  6 shows  the  descriptive 
statistics  for  the  effort  and  involvement  of  students  whose  learning  style  matched  the  instructional  strategy. 

Table  5.  Results  of  One  way  Analysis  of  Variance  for  Effort  and  Involvement  of  Students  whose  Learning  Style 
Matched  the  Instructional  Strategy  Used  to  Present  the  Online  Course  Module 


Source 

Sum  of  Squares 

D.F. 

Mean  Squares 

F ratio 

F Prob. 

Between  groups 

1.06779522 

2 

.533897609 

1.02 

0.3795 

Within  groups 

9.39358058 

18 

.521865588 

Total 

10.4613758 

20 

.52306879 

Table  6.  The  Descriptive  Statistics  for  the  Effort  and  Involvement  of  Students  whose  Learning  Style  Matched  the 
Instructional  Strategy 


Matched  Group 

Instructional  Strategy 

Mean 

N 

S.D. 

Min 

Max 

Low  Field  Dependent 

Expository 

3.71 

9 

0.5 

3 

4.4 

Low  Field  Dependent 

Collaborative 

3.81 

9 

0.64 

3 

5 

High  Field  Dependent 

Discovery 

3.13 

3 

1.41 

1.6 

4.4 

The  third  null  hypothesis  stated  that  there  would  be  no  significant  difference  in  the  perceived  level  of 
interaction  of  students  whose  learning  style  matched  the  instructional  strategy.  The  results  of  the  one  way  analysis  of 
variance  supported  this  hypothesis,  F(2,18)  = 0.03,  p = 0.97  (see  Table  7).  No  significant  difference  was  found  in  the 
perceived  level  of  interaction  of  students  whose  learning  style  matched  the  instructional  strategy.  Low  and  high  field 
dependent  students  perceived  their  level  of  interactivity  same  for  all  three  types  of  instructional  strategies  used  in 
these  modules.  Table  8 shows  the  descriptive  statistics  for  the  level  of  interaction  perceived  by  students  whose 
learning  style  matched  the  instructional  strategies. 

Table  7.  Results  of  One-way  Analysis  of  Variance  for  Perceived  Level  of  Interaction  of  Students  whose  Learning 
Style  Matched  the  Instructional  Strategy  Used  to  Present  the  Online  Course  Module 


Source 

Sum  of  Squares 

D.F. 

Mean  Squares 

F ratio 

F Prob. 

Between  groups 

.054603198 

2 

.027301599 

0.03 

0.9703 

Within  groups 

16.2755553 

18 

.904197518 

Total 

16.3301585 

20 

.816507926 

Table  8.  The  Descriptive  Statistics  for  the  Level  of  Interaction  Perceived  by  Students  whose  Learning  Style  Matched 
the  Instructional  Strategies 


Matched  Group 

Instructional  Strategy 

Mean 

N 

S.D. 

Min 

Max 

Low  Field  Dependent 

Expository 

3.82 

9 

0.92 

2.6 

5 

Low  Field  Dependent 

Collaborative 

3.77 

9 

0.95 

2.66 

5 

High  Field  Dependent 

Discovery 

3.66 

3 

1.0 

2.6 

4.6 

One  Superior  Instructional  Strategy 

The  fourth  null  hypothesis  stated  that  there  would  be  no  significant  difference  in  the  perceived  learning 
outcomes  for  low-field-dependent  learners  in  match  and  mismatch  instructional  strategy  situations.  The  results  of  the 
one  way  analysis  of  variance  supported  this  null  hypothesis,  F(2,24)  = 0.19,  p = 0.82  (see  Table  9).  No  significant 
difference  was  found  in  the  perceived  learning  outcomes  of  low-field-dependent  students  who  completed  three 
online  course  modules.  The  characteristics  of  low  field  dependent  students  showed  match  with  expository  and 
collaborative  type  of  instructional  strategies,  and  mismatch  with  discovery  type  of  instructional  strategies.  Statistical 
analysis  showed  no  significant  difference  in  the  perceived  learning  outcomes  of  low  field  dependent  students  in 
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match  and  mismatch  instructional  strategy  situations.  Table  10  shows  the  descriptive  statistics  for  the  perceived 
learning  outcomes  for  low  field  dependent  learners  in  match  and  mismatch  instructional  strategy  situations. 


Table  9.  Results  of  One-way  Analysis  of  Variance  for  Perceived  Learning  Outcomes  of  Low-Field-Dependent 
Students  in  Match  and  Mismatch  Instructional  Strategy  Situations 


Source 

Sum  of  Squares 

D.F. 

Mean  Squares 

F ratio 

F Prob. 

Between  groups 

.234073991 

2 

.117036996 

0.19 

0.8286 

Within  groups 

14.8266668 

24 

.617777781 

Total 

15.0607407 

26 

.579259259 

Table  10.  The  Descriptive  Statistics  for  the  Perceived  Learning  Outcomes  for  Low  Field  Dependent  Learners  in 
Match  and  Mismatch  Instructional  Strategy  Situations 


Low  Field  Dependent 

Learners  Match  and  Mismatch  Situations 

Instructional  Strategy 

Mean 

N 

S.D. 

Min 

Max 

Match 

Expository 

3.71 

9 

0.85 

2 

5 

Match 

Collaborative 

3.55 

9 

0.88 

2 

5 

Mismatch 

Discovery 

3.77 

9 

0.58 

3 

4.8 

The  last  null  hypothesis  stated  that  there  would  be  no  significant  difference  in  the  perceived  learning 
outcomes  for  high-field-dependent  learners  in  match  and  mismatch  instructional  strategy  situations.  The  results  of 
the  one  way  analysis  of  variance  supported  this  null  hypothesis,  F(2,6)  = 0.13,  p = 0.88  (see  Table  11).  No 
significant  difference  was  found  in  the  perceived  learning  outcomes  of  high-field-dependent  students  who  completed 
three  online  course  modules  each  of  which  used  different  instructional  strategy.  Perceived  learning  outcomes  of  high 
field  dependent  students  did  not  change  when  they  were  taught  with  different  instructional  strategies  matching  and 
mismatching  their  characteristics.  Table  12  shows  the  descriptive  statistics  for  the  perceived  learning  outcomes  for 
high  field  dependent  learners  in  match  and  mismatch  instructional  strategy  situations. 

Table  11.  Results  of  One-way  Analysis  of  Variance  for  Perceived  Learning  Outcomes  of  High-Field-Dependent 
Students  in  Match  and  Mismatch  Instructional  Strategy  Situations 


Source 

Sum  of  Squares 

D.F. 

Mean  Squares 

F ratio 

F Prob. 

Between  groups 

.267654316 

2 

.133827158 

0.13 

0.8824 

Within  groups 

6.28740728 

6 

1.04790121 

Total 

6.5550616 

8 

.8193827 

Table  12.  The  Descriptive  Statistics  for  the  Perceived  Learning  Outcomes  for  High  Field  Dependent  Learners  in 
Match  and  Mismatch  Instructional  Strategy  Situations 


High  Field  Dependent 

Learners  Match  and  Mismatch  Situations 

Instructional  Strategy 

Mean 

N 

S.D. 

Min 

Max 

Mismatch 

Expository 

3.66 

3 

0.94 

2.6 

4.4 

Mismatch 

Collaborative 

3.88 

3 

1.01 

3 

5 

Match 

Discovery 

3.46 

3 

1.1 

2.4 

4.6 

Discussion  and  Conclusion 

Delivering  instruction  on  the  Internet  has  become  very  popular  in  recent  years.  Often  face-to-face  courses 
are  converted  to  online  course  activities  and  materials  with  little  thought  of  learners’  preferences  for  instruction. 
Understanding  the  effects  that  learning  styles  and  learners’  perceptions  of  engagement  in  online  environments  have 
potential  to  improve  the  planning,  producing,  and  implementing  of  online  educational  experiences.  Thus,  learning 
styles  can  be  utilized  to  enhance  students’  learning,  retention,  and  retrieval  (Federico,  2000).  This  study  provides 
insight  into  the  relationships  among  learning  style  and  instructional  strategies  used  in  online  environments. 

The  statistical  analysis  revealed  no  significant  differences  among  three  match  situations  for  low  and  high 
field  dependent  students.  When  the  characteristics  of  low  and  high  field  dependent  students  matched  with 
instructional  strategies,  match  groups  did  not  show  any  statistically  significant  difference  in  their  perceived  learning 
outcomes,  their  perceived  effort  and  involvement  in  units,  and  level  of  interactivity  that  they  perceived  during  the 
unit.  This  result  showed  that  when  low  and  high  field  dependent  students  receive  instruction  utilizing  instructional 
strategies  matching  their  characteristics,  they  gain  equal  learning  benefits  from  the  instruction.  Using  expository  and 
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collaborative  type  of  instructional  strategies  for  high  field  dependent  students,  and  using  discovery  type  of 
instructional  strategies  for  low  field  dependent  students  in  online  courses  provided  equal  benefits  for  students  in 
terms  of  their  perceived  learning  outcomes,  their  perceived  effort  and  involvement,  and  level  of  interactivity  that 
they  perceived  in  the  class.  However,  considering  the  fact  that  mean  scores  of  students  for  match  situations  were 
more  than  the  average  score,  matching  instructional  strategies  with  low  and  high  field  dependent  learners  appears  to 
show  some  positive  effect  on  student  learning.  Online  course  instructors  may  utilize  expository  and  collaborative 
types  of  instructional  strategies  for  high  field  dependent  students,  and  discovery  types  of  instructional  strategies  for 
low  field  dependent  students  to  make  the  instruction  more  appealing  and  effective.  Ultimately  online  students  may 
gain  more  learning  benefits  from  the  course  in  terms  of  their  perceived  learning  outcome,  their  effort  and 
involvement,  and  level  of  activity  that  they  perceive  in  the  online  class. 

The  results  also  revealed  that  there  is  no  single  superior  instructional  strategy  for  high  and  low  field 
dependent  students  among  the  three  types  of  instructional  strategy  used  in  the  study.  The  characteristics  of  low  field 
dependent  students  matched  expository  and  collaborative  instructional  strategies  and  mismatched  discovery  type  of 
instructional  strategies.  When  low  field  dependent  student  groups  were  statistically  compared,  no  significant 
differences  were  detected  for  three  constructs  used  in  the  study.  Matching  and  mismatching  instructional  strategies 
for  low  field  dependent  students  did  not  affect  students’  perceived  learning  outcome,  their  perceived  effort  and 
involvement  in  units,  and  level  of  interactivity  that  they  perceived  during  the  unit.  Similar  statistical  analysis  was 
conducted  for  high  field  dependent  students  whose  characteristics  matched  discovery  type  of  instructional  strategies 
and  mismatched  expository  and  collaborative  type  of  instructional  strategies.  However,  statistically  no  significant 
results  were  found  for  high  field  dependent  students  as  well.  Results  of  this  study  showed  that  utilizing  expository, 
collaborative,  and  discovery  types  of  instructional  strategies  to  design  online  courses  provided  almost  equal  learning 
benefits  for  low  and  high  field  dependent  students. 

Although,  this  pilot  study  provided  valuable  information  on  gathering  learner  style  information  from  online 
learners,  results  of  the  study  should  be  interpreted  with  caution.  These  findings  may  have  been  due  to  a number  of 
factors.  Finding  no  significant  results  could  have  been  due  to  small  number  of  subjects.  Considering  the  fact  that 
there  were  twelve- subjects  involved  to  the  study  and  only  three  subjects  were  categorized  as  high- field  dependent 
individuals,  more  subjects  are  required  to  validate  the  results  of  this  pilot  study.  There  appears  to  be  other  factors 
that  may  have  affected  the  results  of  the  study.  Existing  course  structure  may  not  have  provided  pure  experiences  in 
different  instructional  strategies.  Furthermore,  the  time  allocated  to  complete  units  was  not  same  so  it  may  have 
influenced  the  experiences  of  students  in  three  units.  Finally  the  content  of  units  were  different  so  the  content  may 
have  influenced  the  level  of  effort  that  each  student  put  into  completing  units. 

Future  researchers  should  consider  testing  environments  that  do  strictly  follow  instructional  strategy 
guidelines  to  confirm  these  findings.  Researchers  should  also  consider  testing  other  learning  style  instruments  and 
instructional  strategies  in  their  future  research.  Although  no  significant  differences  were  identified  in  this  study, 
there  is  much  to  learn  about  how  individuals  interact  and  learn  in  online  environments. 
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Capturing  Rehearsals  to  Facilitate  Reflection 
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Introduction 

Many  learning  environments  involve  rituals  for  rehearsal  and  reflection.  Musicians,  for  instance,  spend 
countless  hours  practicing  scales  and  adjusting  their  bodies  to  increase  their  skills.  But  they  do  more  than  simply 
practice:  They  also  play  for  instructors  and  others  who  can  provide  valuable  critiques  of  their  performances. 
Architectural  design  studios  encourage  students  to  create  designs  and  share  them  with  experts  and  peers  in  organized 
“crit”  sessions  that  point  out  good  and  bad  aspects  of  their  work.  Athletic  coaches  often  watch  videos  of  games  with 
their  players  to  reflect  on  issues  for  improvement.  In  all  cases,  there  is  a cycle  of  skill  rehearsal  followed  by  periods 
of  critical  reflection  to  understand  successes  and  failures,  ultimately  to  improve  future  performance. 

Much  of  reflection  is  about  making  tacit  knowledge  and  routines  explicit  so  they  can  be  analyzed  and 
promote  self-awareness  or  “knowledge-in-action”  (Lin  et  al.,  1999;  Schon,  1983).  In  the  above  examples,  these 
reflections  are  partially  facilitated  through  concrete  artifacts  that  capture  aspects  of  past  performance.  Musical 
sessions  can  be  recorded  to  tape,  architecture  students  create  drawings  and  models,  and  athletes  use  video  when 
reflecting  on  their  skills.  Otherwise  transient  actions  and  performances  are  captured  and  made  explicit  as  concrete 
artifacts  for  reflective  thinking  and  learning. 

Our  research  considers  the  importance  of  making  actions  into  artifacts  for  reflective  thinking.  In  particular, 
we  will  describe  ongoing  efforts  to  develop  computer-based  visualizations  for  diabetes  health  management. 
Approximately  17  million  American  suffer  from  diabetes  (NIDDK,  1998),  and  those  numbers  continue  to  increase. 
The  disease  cannot  be  cured,  but  it  can  be  managed  through  insulin  and  oral  medications  and  changes  in  diet  and 
exercise  habits.  We  are  focused  on  the  latter  part  of  diabetes  self-management,  the  regulation  of  daily  routines  to 
prevent  abnormal  blood  sugar  levels  that  could  lead  to  future  health  complications. 

Most  diabetics  carry  and  use  glucose  meters,  small,  handheld  devices  that  measure  and  report  current  blood 
sugar  levels.  These  technologies  are  critical  to  diabetic  lifestyles,  as  they  present  physiological  data  to  help  people 
see  how  they  are  dealing  with  the  disease.  Our  research  tries  to  add  additional  information  to  glucose  meters  by 
helping  diabetics  explore  questions  about  why  their  sugar  levels  are  normal  or  abnormal  during  the  day. 

Specifically,  we  developed  a computer-based  visualization  for  displaying  glucose  meter  data  that  makes  patterns  of 
regularity  (or  irregularity)  explicit  to  its  users.  The  hallmark  of  these  visualizations  is  the  use  of  color  to  provide 
global  overviews  of  high,  low,  and  normal  blood  sugars  over  extended  periods  of  time. 

Beyond  visualizing  physiological  data,  we  have  diabetics  take  photographs  of  their  daily  activities, 
focusing  on  things  that  might  impact  their  blood  sugar  levels.  These  images  are  integrated  into  the  computer 
visualizations  to  contextualize  the  numerical  data.  Our  hypothesis  was  that  diabetics  could  begin  to  engage  in 
reflective  thinking  around  their  health  practices  when  provided  with  visualizations  that  point  out  potential 
correlations  between  blood  sugar  levels  (captured  by  glucose  meters)  and  behaviors  (captured  in  photographs). 

We  will  report  results  from  a recent  study  of  the  use  of  visualizations  of  behavioral  and  physiological  data  to 
enhance  the  aspects  of  reflection  stated  in  findings 

Definitions  of  Reflection 

Reflection  has  been  defined  differently  by  different  people.  Before  analyzing  our  study  data  we  considered 
the  following  definitions. 

In  a chapter  in  How  We  Think  (1997),  entitled  “What  is  Thought?”.  Dewey  defines  and  emphasizes  the 
importance  of  reflective  thought.  Reflective  thought  is  one  of  the  four  senses  of  thought:  the  process  of  accepting  a 
belief  after  deliberately  seeking  and  examining  its  grounds.  Reflection  involves  the  consequences  of  ideas  rather 
than  merely  a sequence  of  ideas.  Each  created  idea  is  a link  in  a chain  of  ideas.  The  important  consequences  of 
beliefs  or  behaviors  might  force  one  to  consider  the  reasons  for  these  and  come  to  a “reasoned  conclusion”. 
Reflective  thought  is  the  “active,  persistent,  and  careful  consideration  of  any  belief  or  supposed  form  of  knowledge 
in  the  light  of  the  grounds  that  support  it,  and  the  further  conclusions  to  which  it  tends”  (Dewey,  p.  6).  Reflective 
thinking  has  two  elements  or  subprocesses:  “a  state  of  perplexity,  hesitation,  doubt”  and  an  investigation  to  confirm 
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or  refute  the  further  facts  (Dewey,  p.  9).  In  other  words  reflective  thinking  starts  with  facing  a problem  or 
questioning  and  then  continues  with  efforts  at  solving  the  problem  by  reasoning.  The  process  ends  with  a reasoned 
conclusion  which  would  be  the  start  of  another  reflective  thinking  process.  Dewey  concludes  that  reflective  thinking 
“. . . means  judgment  suspended  during  further  inquiry;  and  suspense  is  likely  to  be  somewhat  painful.”  (1997,  p.  13) 

Schon’s  definition  of  reflection  comes  with  the  following  terms:  knowing-in-action,  reflection-in- action 
and  reflection  on  action-in-action.  Knowing-in- action  refers  to  the  kind  of  knowing  that  is  revealed  in  our  intelligent 
action  while  executing  a spontaneous  performance  (Schon,  1987).  This  is  a kind  of  action  that  cannot  be  verbalized. 
The  knowing  occurs  in  the  action  as  a response  to  an  unexpected  outcome.  The  unexpected  outcome  can  be  result  of 
anything  that  does  not  happen  as  a part  of  routine.  Further  knowing-in-action  may  help  us  to  reflect  on  the 
unexpected  outcomes  of  daily  routine  and  take  an  in  action.  For  example,  automatically  steering  a bicycle  to  the  left 
in  order  to  maintain  balance  whenever  the  bicycle  tilts  to  the  left  is  an  example  of  reflection-in  action,  The  action 
which  occurs  in  that  present  moment  as  a response  to  the  unexpected  results  from  and  demonstrates  knowing-in- 
action.  During  an  action,  we  still  can  make  a change  to  the  situation  and  our  thinking  serves  to  reshape  what  we  are 
doing  while  we  are  doing.  In  cases  like  this,  we  reflect-in-action,  “thinking  on  our  feet”  (Schon,  1983,  p.  54).  For 
example  one  bikes  to  the  left  naturally  when  the  bike  tilts  to  the  left  at  the  moment  of  in-action.  When  we  think  back 
to  see  how  our  knowing-in-action  helped  us  respond  to  the  unexpected  situation,  then  we  are  reflecting  on  action. 
Reflection  on  action  can  help  reveal  our  own  theories  as  well.  When  one  finds  himself  experiencing  puzzlement,  or 
confusion  in  an  uncertain  or  unique  situation,  “he  reflects  on  the  phenomena  before  him,  and  on  the  prior 
understandings  which  have  been  implicit  in  his  behavior  (Schon,  1983,  p.  68).  This  experience  results  in  generating 
“both  a new  understanding  of  the  phenomena  and  a change  in  the  situation”  (Schon,  1983,  p.  68). 

Self-reflection  and  reflection  are  the  terms  defined  as  a part  of  self-regulated  learning  (SRL)  by 
Zimmerman  and  Pintrich  (Puustinen  & Pulkkinen,  2001).  Zimmerman  (1998)  defines  self-regulated  learning  as  a 
cyclical  process  which  has  three  components:  forethought,  performance  and  self-reflection.  Each  of  these 
components  support  each  other  in  a sequence.  Further  more  specifically  self-reflection  has  four  types  of  processes: 
self-evaluation,  attributions,  self-reactions  and  adaptivity.  As  one  of  the  initial  processes  during  self-evaluation,  one 
compares  self-monitored  information  with  a goal.  Self-evaluation  leads  one  to  attributions  of  reasoned  conclusions. 
Attributions  lead  one  to  self-reaction  and  also  to  adaptation  to  the  performance.  Similarly,  Pintrich’ s SRL  definition 
has  reflection  as  one  of  the  four  phases,  whereas  the  first  three  are  forethought,  monitoring,  control  and  reflection 
(Puustinen  & Pulkkinen,  2001).  More  “reflection  includes  cognitive  judgments,  affective  reactions,  making  choices 
and  task  and  context  evaluation”. (Puustinen  & Pulkkinen,  p.  274)” 

Study  described 

Specifically,  our  study  involved  six  type  I diabetics  and  one  type  II  diabetic.  This  group  took  photographs 
of  their  everyday  activities  for  a month  and  met  with  us  weekly  to  discuss  their  health,  using  their  visualized  data  as 
a conversational  prop  (Brinck  & Gomez,  1992;  Roschelle,  1992)  for  reflection. 

They  were  given  small  digital  cameras  and  asked  to  take  pictures  of  diabetes-related  behaviors.  The 
diabetics  were  free  to  decide  what  they  would  take  pictures  of.  We  suggested  that  they  might  take  pictures  of  their 
meals  and  exercise  activities.  In  their  daily  activities  they  were  testing  and  monitoring  glucose  results  by  using  a 
digital  glucose  tester. 

Before  each  weekly  meeting,  we  uploaded  the  glucose  data  and  the  pictures  to  the  computer.  Then  we 
displayed  the  data  with  software  which  was  created  for  this  purpose  (see  Figure  1 and  2).  With  this  interface  we 
could  see  the  glucose  results  and  diabetes-related  activities,  captured  in  pictures,  accompanied  by  a record  of  the 
time  and  day.  For  the  glucose  results,  color  codes  were  used  in  order  to  facilitate  an  easy  and  quick  grasp  of  glucose 
patterns.  The  colors  represented  ranges  of  blood  sugar  levels  as  follows: 

• Dark  blue:  0-39 

• Lighter  blue:  40-80 

• Gray:  81-120 

• Red:  121-140 

• Dark  Red:  141  and  higher 

The  data  for  the  entire  period  were  displayed  and  then  we  posed  questions  to  facilitate  reflection.  When  necessary, 
the  pictures  were  magnified  (see  Figure  3). 

What  follows  is  a discussion,  based  on  preliminary  data  analysis,  of  the  experience  of  the  participants  and 
how  it  helped  them  reflect. 
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Findings 

Schon  claims  that  his  examples  of  a baseball  pitcher  who  reflects  on  “winning  habits”  and  a jazz  musician 
who  reflects  on  the  experience  of  making  music,  show  that  “reflection  tends  to  focus  interactively  on  the  outcomes 
of  action,  the  action  itself,  and  the  intuitive  knowing  implicit  in  action”  (1983,  p.  56).  In  our  study,  participants  were 
invited  to  focus  on  the  outcomes  of  activities  related  to  being  diabetic,  activities  by  themselves  and  the  intuitive 
knowing  implicit  in  activity.  Before  taking  photos  of  activities  they  had  to  focus  on  the  outcomes  of  the  activities 
related  to  diabetes.  For  example,  how  does  this  exercise  affect  my  glucose?  Most  of  them  took  pictures  of  their 
meals  and  exercise  activities  such  as  walking  and  weight-lifting.  In  the  process  of  capturing  activities  the  participant 
experiences  the  intuitive  knowing  implicit  in  the  activity.  The  participant  makes  this  implicit  knowledge  explicit  in 
two  ways.  The  participant  makes  the  knowledge  explicit  by  communicating  via  visual  representations  of  captured 
activities,  photos  and  also  by  talking  about  them  later  on  during  the  interviews.  The  knowledge  is  received  by  two 
parties:  the  participant  and  the  interviewer  who  in  some  cases  could  be  a health  care  person  or  physician. 

Following  we  will  introduce  our  preliminary  data  results  associated  with  these  phases. 

Participants’  awareness  about  what  to  do  was  extremely  variable.  Most  of  the  participants  know  what  to  do  in 
general  for  their  health  care.  PI 08,  who  was  an  athletic,  had  already  come  to  an  understanding  through  past 
experience  of  what  affects  her  blood  sugar  and  how  to  deal  with  it  by  correlating  her  eating  and  exercise.  She  had 
been  exercising  couple  of  times  per  a week  since  her  childhood.  She  kept  log  books  recording  of  what  she  ate  and 
what  she  did  for  exercise.  She  thinks  she  does  not  have  room  to  improve  much.  In  other  words,  she  does  not  have 
much  confusion  in  her  diabetes-related  life.  However  she  still  is  not  sure  about  some  blood  sugar  changes  that 
specifically  occur  due  to  exercise.  She  still  has  not  found  the  reasons  of  this.  This  is  still  a puzzle  to  her.  When  we 
asked  what  she  learned  about  herself  during  the  study,  she  answered: 

“. . .1  don’t  think  I learned  anything  new. . .because  I am  an  athlete  and  I have  been  at  least  recently  really 
looking  at  my  diabetics  what  affects  it  and  how  to  change  it,  kind  of  understanding  myself  more  what  the  exercise 
does  to  my  health  aham  I think  I have  already  gone  that  process  but  if  maybe  the  other  people  haven’t  aham  that 
they  don’t  really  understand  what  affects  it  as  much  it  might  help  them  for  a better  understanding.  Since  I wrote 
down  everything  I eat  what  insulin  I take,  I can  sort  of  already  look  back  and  see  what  affects  more. . .1  have  a basic 
understanding  but  some  things  are  still  different  like  I said  exercise  will  kick  in  at  strange  times,  some  times  directly 
after  sometimes  later  which  I am  still  trying  to  figure  out. . .for  some  reason  one  day  my  body  would  kick  in  earlier 
or  later”  (P108,  interview4,  conscious,  1:36) 

On  the  other  hand  some  of  the  participants  still  did  not  know  what  causes  what.  They  were  still  in 
confusion  or  hesitation.  And  knowing  the  reasons  for  changes  could  be  helpful  for  PI 06. 

“. . .1  hope  to  see  what  it  is  causing  me  to  get  high  blood  sugar. . .but  if  I see  what  is  causing  me  that  would  definitely 
be  direction...”  (P106,  interviewl,  conscious,  20:03) 

Even  though  they  know,  what  they  do  or  they  do  not  do  to  manage  their  diabetes  has  already  become  part 
of  their  daily  routine.  They  may  not  be  thinking  about  their  diabetics -related  experiences  enough  because  they  have 
become  repetitive  practices  embedded  in  their  daily  routines.  For  example,  Schon  (1983,  p.  61)  mentions  that  when 
practice  becomes  more  repetitive  and  routine,  knowing  becomes  sufficiently  implicit  and  natural  that  the  practitioner 
may  miss  important  opportunities  of  careful  consideration  of  what  he  is  doing.  By  asking  participants  to  capture 
their  diabetes-related  behaviors,  we  invited  them  stand  outside  of  their  routine  lives  and  think  actively,  persistently 
and  carefully  about  what  they  are  doing  for  their  health  care  as  they  are  doing  it  (thinking  on  their  feet).  As  a result 
of  experiencing  their  lives  out  of  their  routine  by  wearing  different  lenses  which  comes  with  the  study  requirement, 
capturing  behaviors,  they  could  reveal  their  knowing-in-action  and  then  reflect  in-action. 

For  example  PI 09  was  not  sure  about  the  reasons  for  his  high  glucose  measures  and  he  was  having 
suspended  judgment: 

“. . .1  think  with  this  really  high  one  I had  a big  dinner  that  evening  that  could  have  been  of  the  causes  of 
that. . .aham  I know  for  most  of  the  high  ones  they  come  after  meals. . .that  could  be  something  to  do  with  eating  a 
meal  right  before  that  that  could  be  why  they  are  high. . .”  (P109,  interviewl,  conscious,  24:58) 
and  he  was  reflecting-in-action  by  facing  the  problem  that  needs  to  be  fixed  but  not  having  the  solution/  response 
yet: 

“. . .1  think  now  looking  at  the  blood  sugars  that  there  are  a lot  of  higher  ones  which  something  needs  to  be 
fixed...”  (P109,  interviewl,  conscious,  27:02) 

While  monitoring  the  two-week  data  on  our  second  interview,  he  continues  reflection-in-action, 
advantageously,  this  time  he  realizes  some  patterns  on  his  high  glucose  results  in  the  mornings  and  questions  the 
reason.  He  wants  to  know  the  reason.  This  is  still  reflection-in-action,  investigation,  since  there  is  an  inquiry  but  he 
does  not  respond  to  the  problem  yet. 


20 


. .1  think  I need  to  trying  get  my  blood  sugar  under  control  especially  in  the  mornings.  I have  to  figure  out 
what  is  causing  that.”  (PI 09,  interview2,  conscious,  56:37) 

PI 07  started  living  a different  life  when  she  left  home  and  became  a college  student.  This  new  life  brought 
her  some  perplexity  and  suspended  judgment  regarding  her  diabetes  management.  We  can  see  her  experience  of  four 
phases  at  the  same  time:  doubt,  context  evaluation,  making  choices  and  changes. 

“. . .(her  doctor)  said  they  should  have  nutrition  facts  up  there  stuff,  they  do  have  that  for  a lot  stuff  but  I eat 
a lot  at  the  salad  bar  and  they  don’t  have  that  stuff  up  so  unless  I go  and  bug  somebody  in.  I tried  to  looking  up  on 
the  Internet. . .now  just  trying  judging. . .now  I am  trying. . .1  better  go  low  then  high  but  it  is  hard  to  the  fact  that  at 
home  I think  I could  manage  my  diabetes  better. . .here  my  schedule  I mean  everything  is  so  different  every  day  is 
different  and  I am  walking  everywhere  so  that’s  why  it  is  much  easier  to  control  at  home. . .”  (PI 07,  interview  1,  as  a 
student-nutrition,  16:29) 

“. . .1  have  been  a lot  more  conscious  about  how  to  cover  my  meals. . .they  don’t  have  carbs  in  the  dining 
hall. . .so  I am  trying  to  guess  and  it  is  the  reason  why  sometimes  it  is  high  or  low. . .so  I have  been  trying  to  eat  more 
regular  salad  dressing  not  fat-free”  (P107,  interviewl,  conscious,  04:17) 

Choosing  the  regular  salad  dressing  shows  her  solution.  She  has  a problem  and  responds  to  that  problem 
while  it  is  occurring.  She  is  reflecting-in-action  and  also  making  choices  and  changes. 

Monitoring  the  glucose  data  and  captured  activities  supported  the  participants  on  their  investigation.  For 
example  both  PI 09  also  PI  10  observe  how  their  eating  amounts  affect  their  blood  sugars  by  monitoring  the  patterns 
on  the  visualized  and  color-coded  data.  They  noticed  that  their  glucose  level  was  high  in  the  mornings  as  well. 

More,  some  of  them  stated  some  new  understandings  of  their  life: 

“..It  was  interesting  to  see  what  kind  of  habits  I have.  Maybe  I realized  what  I should  do  a little  bit 
more. . .Like  I really  never  thought  about  sleeping  how  it  affects  my  sugar. . .”  (P105,  interview4,  conscious,  36:27) 

“. . .this  is  interesting  because  I don’t  usually  view  as  stress  affecting  me  much  but  you  guys  said  anything 
affect  my  blood  sugar  level  that  I could  feel  it. . .like  affecting  me  and  I don’t  usually  give  that  much”  (PI 06, 
interviewl,  conscious,  10:14) 

“. . .1  just  became  more  aware  like  at  the  gym. . .this  is  first  time  I kept  a log  book  for  a long  time  it  kind  of 
helps  seeing  the  pattern... ”(P  107,  interview3,  conscious,  12:21) 

Monitoring  also  helped  for  realizing  some  habits,  self-evaluation  and  followed  with  confirming  the  fact: 

“. . .1  took  a picture  of  glucose  tablets. . .aham  let  me  see. . .geez  I don’t  why  I ate  glucose  tablets.  I just  eat 
them  sometimes  when  I feel  low. . .well  that’s  even  normal  to  deal  with  . . .we  (then  he  sees  the  pictures  and  says) 
yeah  I am  actually  starting  to  remember  this  day  I ate  a lot. . .1  ate  what  ever  I want. . .then  I try  to  compensate  with 
extra  insulin  which  is  not  good  idea  I am  trying  to  learn  more  on  that,  me  myself. . .”  (PI 06,  interview3,  conscious, 
30:30) 

After  investigation  confirmation  emerged.  Some  participants  noticed  or  confirmed  that  different  kinds  of 
exercising  such  as  walking  versus  weight  lifting  affects  blood  sugar  drops  differently.  For  example  PI 08  mentioned 
that  weight  lifting  kind  of  activities  affect  later  rather  than  just  immediate  after.  She  already  had  an  awareness  of 
that.  On  the  other  hand  PI 09  was  not  sure  of  this  and  had  an  idea  about  that  when  questions  prompted  him  to  see  a 
correlation: 

“. . .maybe  weight  lifting  is  affecting  the  next  day  more  than  the  same  day. . .”  (P109,  interview4,  conscious, 

45:47) 

“. . .1  think  we  saw  a connection  between  exercise  and  the  next  day  blood  sugar  seems  to  be  lower  in  the 
morning  the  day  after  exercise.  I think  it  is  one  pattern  that  we  saw.  Seems  that  if  I have  a big  dinner  I don’t  know 
how  to  adjust  (?)... ”(P109,  interview4,  conscious,  51:05) 

When  asked  to  provide  the  reasons  for  the  fluctuations  by  monitoring  the  data,  one  participant  responded, 

“. . .Aham  there  could  be  a lot  of  different  reasons  of  that.  For  instance  if  I am  taking  the  insulin  or  I am 
taking  too  much  insulin  or  eating  I have  noticed  depending  on  what  eat  has  different  effects,  like  if  I eat  pizza  or 
something  the  effects  of  that  don’t  really  impact  me  until  like  two  hours  later.”  (PI  11,  interview  3,  measures- 
fluctuations,  36:46) 

In  the  fourth  interview  when  the  color-coded  data  viewed  Pill  realizes  changes: 

“. . .1  see  a significant  improvement  in  the  back  last  five  days. . .” 
and  then  he  comes  to  some  conclusions  when  he  is  asked  to  see  the  connection  between  his  exercise  and  blood 
glucose  data: 

Interviewer  1:  “. . .when  you  look  all  overall  this  data  and  do  you  see  connection  between  your  exercise  and 
blood  glucose  data?” 

Pill:  “definitely” 

Interviewer  1:  “how?” 
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. .aham  I think  (?)  last  week  (?)  shows  and  from  when  I work  as  well  shows  that  my  blood  glucoses 
decreases  dramatically  so  if  it  doesn’t  decrease  dramatically  immediately  if  it  doesn’t  immediate  effect  like  from 
running  or  something  aham  symptoms  of  it  will  definitely  show  up  in  the  morning  rather  than  at  night  and  from  I 
think  from  work  since  work  is  usually  a longer  period  than  a few  hours  you  see  the  results  of  working  of  when  I 
finish  working  my  work  I test  my  blood  and  you  see  the  results. . (PI  11,  interview  4,  reflection-on,  22:48) 

Ultimately,  some  participants  reported  some  changes  or  considerations  of  changes  in  their  life.  After  they 
responded  to  the  problem  and  then  reflected  on  that.  Some  participants  increased  their  exercise  with  more  walking 
or  decreased  eating  non-recommended  food.  For  example  Pill  stated  that  he  stopped  eating  ice  cream,  or  PI  10  tried 
to  increase  his  exercise  (walking)  and  decided  not  to  eat  late  at  night.  PI  1 1 is  talking  on  the  experience  of  the  study: 
“. . .it  was  definitely  a positive  experience  just  because  of  the  fact  that  I just  had  to  be  consciously  aware  of  yeah  I 
need  to  take  picture  of  this  and  sometimes  it  is  a horrible  (deter?)  but  sometimes  I think  that  ‘do  I really  need  to  eat 
this?’because  I am  gonna  take  picture  of  it.  I think  it  it  definitely  helped  just  making  me  conscious  of  what  I eat. . .” 
(Pill,  interview4,  DPS-taking  pictures- stops  more  eating,  31:19) 

Interviewer  1 follows:  “Do  you  think  it  affected  your  choices  of  eating?” 

“. . .yeah  a few  times  it  did. . .like  for  instance  this  whole  past  last  two  months  like  I have  not  eaten  any  ice-cream 
which  is  very  weird  considering  that  I love  ice-cream  and  I am  in  the  Penn  State  (referring  the  popular  ice-cream 
place)...”  (Pill,  interview4,  DPS-taking  pictures-stops  more  eating,  31:19) 

“. . .1  have  improved  the  numbers  (blood  results). . .it  is  definitely  worked  a lot. . .”  (P106,  interview4, 
conscious,  43:31) 

Interviewer  1 asks:  “This  experience  have  helped  your  health?” 

“. . .this  just  made  me  realize  I am  pretty  out  (?)  of  control  now. . .so  many  little  things  make  big 
differences. . .1  am  trying  to  get  under  control  I guess. . .”  (P107,  interview4,  conscious,  4:00) 

Limitations  and  Conclusion 

One  of  the  limitations  was  the  interviewers’  lack  of  expertise  on  the  content.  The  interviewers  were  not 
health  experts,  so  that  could  have  decreased  their  ability  to  see  the  correlations  between  the  data  and  the  disease. 
Further,  some  questions  asked  by  the  interviewers  to  facilitate  seeing  the  impact  and  correlation  could  affect  the 
statements  given  by  the  participants.  More  the  tendency  of  pictures  on  the  subjects  of  eating  and  exercise  could  have 
been  the  result  of  examples  given  by  the  researchers  at  the  beginning  of  the  study.  In  addition  to  these,  we  had  some 
technology-related  limitations.  The  camera  was  forgotten  or  inconvenient  to  carry  for  some  participants.  One 
participant  mentioned  that  it  would  be  better  to  have  camera  and  the  monitor  together  in  one  tool,  since  she  already 
has  to  carry  the  monitor.  Also  a few  comments  were  made  about  how  difficult  it  is  to  communicate  everything  by 
taking  pictures.  One  participant  found  writing  log  books  easier  while  some  others  found  taking  pictures  easier. 

Further  applications  and  research  of  this  study  might  be  in  various  areas  for  various  purposes.  In  general, 
this  could  be  used  for  reflection  processes  as  a path  to  improvement.  Facilitating  active,  persistent  and  careful 
consideration  of  lived  experiences  by  thinking  about  captured  and  monitored  activities  might  help  one  to  improve. 
We  can  see  applications  in  health  for  complex  disease  diagnosis,  adaptation  and  management.  The  study  also  has 
broader  educational  applications  because  it  describes  and  evaluates  a form  of  self-regulated  learning. 
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Figure  1 : Glucose  results  on  color-coded  chart 
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Figure  2:  Glucose  results  and  pictures  taken 
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Figure  3:  A magnified  view  of  a picture 
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Communities  of  Practice  as  Organizational  Knowledge  Networks 

Stephanie  Allen 
Brigham  Young  University 

Abstract 

As  the  amount  of  critical  information  in  companies  continues  to  burgeon  and  employees’  knowledge  is 
heralded  as  an  organization ’s  key  competitive  advantage,  knowledge  management  has  become  a compelling 
workplace  topic  of  discussion.  Communities  of  practice  have  recently  been  recognized  as  effective  means  for 
organizations  to  manage  their  knowledge.  In  order  to  determine  how  virtual  communities  of  practice  serve  as 
knowledge  management  vehicles,  the  authors  of  this  paper  conducted  a study  on  virtual  communities  of  practice 
within  twelve  large,  international  companies  by  interviewing  virtual  community  of  practice  builders  and  leaders 
within  these  organizations.  This  paper  reveals  the  study’s  findings.  It  outlines  specific  ways  that  organizations  can 
benefit  from  sponsoring  virtual  communities  of  practice.  It  also  identifies  factors  that  community  builders  and 
leaders  can  influence  to  ensure  a VCoP  ’s  success  as  a knowledge  management  vehicle. 

Introduction 

In  our  current  age  where  information  and  knowledge  are  recognized  as  key  ingredients  for  success  and 
competitive  advantage  (Goldwassar,  2001),  employees’  knowledge,  skills,  and  ideas  are  often  considered  to  be 
companies’  most  valuable  assets  (Schwen,  Kalman,  Hara,  & Kisling,  1998;  Stewart,  1997).  As  the  pressure  to 
manage  knowledge  has  become  more  and  more  pronounced,  companies  are  undertaking  major  initiatives  to  protect 
and  preserve  it  (Wenger,  1998).  Accordingly,  much  literature  has  been  dedicated  to  knowledge  management  (Blunt, 
2001;  Davenport  & Prusak,  1998;  Hamel  & Prahalad,  1994;  Kelly,  1998;  Schwen  et  al.,  1998;  Sharp,  1997;  Pfeffer 
& Sutton,  2000;  Wenger  & Snyder,  2000). 

Proponents  of  knowledge  management  primarily  recommend  two  ways  of  capturing  and  managing  an 
organization’s  knowledge  (Hansen,  Nohria,  & Tierney,  2001;  Schwen  et  al.,  1998).  One  recommended  way  to 
manage  knowledge  is  to  codify,  index,  and  warehouse  the  information  (Davenport  & Prusak,  1997;  Davenport  & 
Prusak,  1998;  Nonaka  & Takeuchi,  1995;  Schwen  et  al.,  1998).  This  method  is  known  as  the  codification  means  of 
managing  knowledge  (Hansen  et  al.,  2001).  Another  recommended  way  to  manage  knowledge  is  to  have  the  people 
who  generate,  refine,  share,  distribute,  and  use  the  knowledge  actively  manage  it  (Brown  & Gray,  1995;  Davenport 
& Prusak,  1998;  Pfeffer  & Sutton,  2000;  Wenger,  2000).  This  method  is  known  as  the  personalization  means  of 
managing  knowledge  (Hansen  et  al.,  2001).  A number  of  individuals  who  promote  actively  managing  knowledge  via 
people  recommend  using  informal  workplace  learning  networks,  like  communities  of  practice  as  knowledge 
management  vehicles  (Masterson,  2002;  Pfeffer  & Sutton,  2000). 

This  paper  describes  and  discusses  the  findings  of  a study  that  reveals  specific  ways  that  virtual 
communities  of  practice  can  benefit  individuals  and  organizations  by  helping  them  manage  their  knowledge  more 
efficiently.  To  place  the  research  in  context,  communities  of  practice  are  first  introduced  and  discussed  as 
knowledge  management  vehicles.  An  account  of  the  research  methodology  follows,  along  with  a general  description 
of  the  study  findings.  The  findings  are  then  discussed  in  detail  in  terms  of  gains  that  individuals  and  organizations 
realize  from  participating  in  and  sponsoring  VCoPs.  The  paper  concludes  with  suggestions  of  how  this  research 
contributes  to  the  field. 


Communities  of  Practice  as  Knowledge  Management  Vehicles 

Communities  of  practice  (CoPs)  are  comprised  of  members  who  communicate  one  with  another  to  generate 
and  share  knowledge  and  expertise.  They  function  as  an  interdependent  network  over  an  extended  period  of  time, 
with  the  shared  goal  of  furthering  their  ‘practice’  or  doing  their  work  better  (Wenger,  1998).  Many  CoPs  operate 
virtually  because  community  members  live  and  work  around  the  globe,  relying  on  various  technological  means  to 
communicate  with  one  another.  These  CoPs  are  often  called  virtual  communities  of  practice  (VCoPs). 

Existing  research  on  communities  of  practice  indicates  that  they  have  been  found  to  help  employees 
acculturate  themselves  into  an  organization  (Chao,  2001;  Gregory,  1993)  develop  a work-associated  identity  (Hara, 
2000;  Yi,  1999),  teach  them  skills  to  get  their  work  done  more  efficiently  (Brown  & Gray,  1995;  Sharp,  1997), 
motivate  them  to  do  their  work  (Barab  & Duffy,  2000;  Bradsher  & Hagan,  1995),  and  improve  their  individual  job 
performance  (Allen,  2003).  However,  literature  relating  the  individual  and  organizational  impact  of  CoPs  to 
knowledge  management  is  limited. 
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Wenger  and  Snyder  (2000)  reported  that  CoPs  have  helped  several  organizations  improve  their  overall 
performance,  enhance  their  communication  structure,  and  support  their  goals.  However,  beyond  a few  individual 
case  studies  and  Wenger  and  Snyder’s  case  study  synthesis,  little  research  focuses  on  the  ways  that  CoPs  facilitate 
knowledge  management  within  organizations.  In  order  to  learn  more  about  the  ways  that  CoPs,  and  particularly 
VCoPs,  help  organizations  and  individual  VCoP  members  manage  their  knowledge,  a study  was  conducted  on 
VCoPs  within  large  corporations,  government  organizations,  and  educational  institutions. 

Research  Methodology 

The  research  team  utilized  interviews  as  the  primary  research  data  source  because  they  are  a key  means  of 
gathering  qualitative  case  study  data  (Stake,  1995;  Yin,  1994).  They  used  a survey  as  a secondary  research  data 
source.  The  researchers  collected  the  study  data  in  two  phases.  They  interviewed  VCoP  builders  and  leaders  in  phase 
one.  During  phase  two,  the  researchers  collected  data  from  VCoP  participants  via  a web-based  survey.  Prior  to 
starting  each  phase,  the  researchers  pilot  tested  and  validated  the  data  collection  instruments  with  a small  group  of 
selected  potential  participants  to  ensure  that  the  collected  data  would  be  reliable  and  valid. 

The  research  participants  involved  in  phase  one  consisted  of  twenty-five  builders  and  leaders  of  virtual 
communities  of  practice  from  thirteen  organizations.  These  organizations  were  primarily  large,  global  companies 
who  provide  a wide  range  of  products  and  services,  such  as  financial  services,  microprocessor  chips,  technology 
services,  training,  health  care  services,  consumer  products,  and  insurance.  The  interviewees  were  carefully  screened 
before  they  were  interviewed  to  ensure  that  the  virtual  communities  of  practice  they  lead  or  built  adhered  to  the 
theory  and  practices  of  community  of  practices.  Once  the  VCoPs’  legitimacy  was  established,  formal  interviews 
were  conducted  via  60-90  minute  telephone  conversations.  These  interviews  were  recorded  and  later  transcribed 
The  interviews  addressed  all  aspects  of  VCoPs  in  order  to  gain  a holistic  understanding  of  their  utilization.  Some 
interview  questions  focused  around  discovering  why  VCoPs  exist  within  organizations,  what  kind  of  “work”  they 
do,  and  how  they  get  started.  Other  questions  attempted  to  ascertain  how  VCoPs  impact  the  flow  of  data, 
information,  and  knowledge  throughout  their  organizations.  For  example,  interviewees  were  asked  to  identify  and 
explain  how  the  communities  of  practice  served  as  forums  to  create  and  disseminate  knowledge  throughout  their 
organizations. 

During  the  second  phase,  the  research  team  used  the  data  gathered  in  phase  one  to  create  a web-based 
survey  that  was  distributed  via  e-mail  to  participants  of  VCoPs  in  forty  organizations.  Approximately  150  virtual 
community  of  practice  members  responded  to  the  web-based  survey.  The  survey  respondents  worked  for  financial 
institutions,  microprocessor  chip  manufacturers,  technology  services  companies,  training  providers,  health  care 
providers,  consumer  product  developers  and  distributors,  insurance  providers,  and  various  other  corporations.  The 
survey  questions  focused  around  discovering  why  participants  join  and  participate  in  VCoPs,  what  factors  contribute 
to  a VCoP’s  success  or  failure,  what  learning  medium  participants  prefer  for  various  types  of  learning  and 
information  exchange,  and  how  learning  in  VCoPs  compares  to  learning  via  more  formal  training  methods.  The 
web-based  survey  software  automatically  captured  the  participants’  responses. 

Findings 

In  general,  the  findings  indicate  that  individuals  and  organizations  can  receive  substantial  knowledge 
management-related  benefits  from  participating  in  and  sponsoring  VCoPs.  The  interviews  with  VCoP  builders  and 
leaders  and  survey  responses  uncovered  a wide  range  of  explicit  and  tacit  knowledge  about  virtual  communities  of 
practice.  The  interviewees  articulated  how  the  VCoPs  actually  perform  in  applied  contexts  and  explained  the 
benefits  they  provide  to  their  members  and  to  the  organizations  that  sponsor  them.  The  survey  data  revealed  ways 
that  members  believe  VCoPs  can  significantly  help  individuals  learn,  manage  their  knowledge,  and  perform  their 
jobs  better. 

Ways  that  VCoPs  Benefit  Individual  Members 

During  the  course  of  the  study,  the  data  provided  both  by  VCoP  builders  and  participants  suggest  that 
VCoPs  benefit  members  by  helping  them  do  their  jobs  better,  support  their  learning  process,  and  extend  or  share 
their  knowledge  with  one  another.  In  their  interview  responses,  builders  of  various  virtual  communities  explained 
how  VCoPs  support  their  members’  learning  processes.  For  example,  one  VCoP  builder  stated: 

[VCoPs]  form  because  that’s  how  people  actually  learn.  That’s  how  they  share  information,  share  resources. . . ask 
questions,  get  answers  to  those  questions,  and  so  forth.  This  is  how  people  really  do  work  and  we  in  the  training 
world,  since  that  is  where  I’m  coming  from,  we  often  look  from  an  educational  training  perspective  that  most 
learning  goes  on  in  the  classroom,  but  the  reality  is  obviously  that  it  doesn’t.  Most  of  it  goes  on  in  the  workplace. . . 
most  of  the  real  critical  learning  goes  on  in  the  workplace. 
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In  general,  the  study  data  suggest  that  VCoPs  help  employees  learn  by  situating  learning  in  the  workplace,  providing 
just-in-time  and  context- specific  solutions  to  problems,  and  increasing  employee  interaction. 

VCoPs  Situate  Learning  in  the  Workplace 

Recent  research  shows  that  employees  often  learn  more  in  the  workplace  than  they  do  in  formal  training 
environments  because  they  fail  to  transfer  and  implement  formal  training  into  their  jobs  (Gilbert,  1978;  Mager, 

1992;  Sorohan,  1993;  Stolovitch  & Keeps,  1999).  VCoPs  are  valuable  on-the-job  learning  environments  because 
they  are  situated  in  employees’  immediate  work  setting.  One  VCoP  builder  stated  that  “a  community  of  practice  is  a 
better  vessel  even  to  do  training  [than  formal  training  environments]  because  it’s  done  within  the  situation  of  the 
work  [they’re]  doing.” 

VCoPs  Provide  Just-in-time  Solutions  to  Problems 

VCoPs  also  provide  a context  where  individual  members  gain  access  to  people  and  resources  that  can  help 
them  solve  their  problems  by  viewing  their  problem  from  multiple  perspectives  and  generating  numerous  problem- 
solving ideas.  In  other  words,  VCoPs  provide  a way  for  members  to  discover  solutions  to  problems  when  they  are 
most  needed  - just  in  time  and  in  their  own  unique  context.  This  significantly  increases  employees’  performance 
capabilities. 

As  part  of  the  web-based  survey,  VCoP  members  were  asked  to  identify  the  benefits  they  personally  gain 
from  participating  in  virtual  communities.  In  response  to  this  question,  99%  of  participant  responded  that  “job  skills 
and  knowledge”  were  the  most  important  benefits.  Approximately  84%  of  participants  also  indicated  that  VCoPs 
provide  “excellent  problem  solving  resources.”  The  percentages  of  these  responses  clearly  suggest  that  people  join 
and  participate  in  VCoPs  to  “gain  knowledge  and  skills”  and  to  “access  resources”  that  will  empower  them  to  make 
better  decisions. 

VCoPs  Increase  Employee  Interaction 

VCoPs  also  benefit  individual  members  because  the  group  structure  of  VCoPs  allows  members  to  share 
information  and  engage  in  learning  activities  with  peers.  The  increased  peer  interaction  then  leads  to  an  increase  in 
the  members’  knowledge  retention  and  stronger  relationships  across  the  organization.  This  is  particularly  true  when 
VCoPs  involve  people  from  across  the  organization,  around  the  world,  and  different  job  descriptions  and 
specializations.  Because  of  these  variations,  the  input  and  connections  made  by  VCoP  members  unite  employees  and 
help  them  gain  a sense  of  purpose  and  awareness  that  their  individual  efforts  contribute  to  organization-wide 
strategies.  In  this  way,  community  participation  and  interaction  reduces  both  hierarchical  and  geographical 
boundaries  and  increases  employee  unification. 

In  a related  survey  question,  VCoP  participants  selected  their  top  three  reasons  for  participating  in  virtual 
communities.  Participants  selected  their  reasons  from  a variety  of  responses  related  to  relationship  building, 
productivity,  status,  and  motives.  In  response  to  this  question,  the  participants  indicated  that  developing 
“professional  relationships  with  other  community  members”  is  their  top  reason  for  participating  in  VCoPs.  This 
response  specifically  relates  to  knowledge  management  because  developing  professional  relationships  with  other 
VCoP  members  is  a key  step  in  transferring  knowledge  among  members.  Also,  when  members  exchange 
information  and  aid  one  another  in  solving  problems,  the  overall  productivity  of  the  community  increases.  This 
increase  in  job-performance  implies  that  employees  are  learning  how  to  do  their  jobs  better. 

The  benefits  of  situating  learning  in  the  workplace,  providing  just-in-time  and  context- specific  solutions  to 
problems,  and  increasing  employee  interaction  were  reinforced  by  a final  question  from  the  web-based  survey, 
which  asked  VCoP  members  to  identify  whether  VCoPs,  web-based/computer-based  training,  instructor-led 
training,  or  mentor/apprentice  learning  environments  best  help  them  accomplish  certain  aspects  of  their  jobs.  In 
response  to  this  survey  question,  VCoP  members  indicated  that  participating  in  VCoPs  helps  them  do  the  following 
six  specified  job  aspects  better: 

• Providing  for  an  efficient  idea  exchange 

• Generating  a broad  perspective  on  solving  problems 

• Providing  greater  access  to  experts 

• Increasing  members’  knowledge 

• Increasing  members’  motivation  to  learn 

• Effectively  helping  others  learn 
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Additionally,  VCoPs  ranked  second  in  conjunction  with  “providing  a more  direct  solution  to  a problem”  and 
“impacting  people’s  attitude  about  their  jobs.”  These  responses  suggest  that  VCoP  members  view  VCoPs  as  a 
learning  environment  that  helps  them  manage  their  knowledge  and  better  perform  on  the  job. 

Ways  that  VCoPs  Benefit  Organizations 

Organizations  also  gain  key  knowledge  management-related  benefits  from  supporting  virtual  communities 
of  practice.  As  previously  indicated,  knowledge  management  initiatives  revolve  around  moving  data,  information, 
and  knowledge  effectively  throughout  an  organization.  The  general  goal  is  to  decrease  the  communication  barriers 
that  exist  in  nearly  every  organization  between  individuals  due  to  divisions,  levels  within  the  organization,  and 
physical  locations.  In  their  interviews,  all  of  the  virtual  community  of  practice  builders  and  leaders  stated  that 
VCoPs  improve  their  organization’s  knowledge  management  initiatives  via  either  direct  or  indirect  means.  They 
indicated  that  virtual  communities  of  practice  facilitate  a greater  flow  of  information  across  organizations  by 
breaking  down  many  of  the  existing  barriers.  They  also  indicated  that  VCoPs  increase  the  networking  and 
communication  opportunities  available  to  VCoP  members  across  organizations  by  providing  increased  interaction 
between  organizational  units  where  communication  was  previously  impossible,  increasing  exchanges  between 
management  and  employees,  extending  discussions  that  occur  in  face-to-face  meetings,  and  creating  a written 
repository  of  best  practices  that  VCoP  members  have  ongoing  access  to.  VCoPs  also  appear  to  increase  the  informal 
training  that  occurs  within  the  organization,  foster  innovation,  and  instigate  cost  savings.  Each  of  these  benefits  is 
elaborated  upon  below. 

VCoPs  Increase  Interaction  Among  the  Best  Minds 

Before  VCoPs  existed,  employees  that  spanned  geographic  borders  or  time  zones  were  limited  in  their 
ability  to  effectively  share  data,  information,  and  knowledge  with  one  another.  Virtual  communities  of  practice 
overcome  time  and  physical  boundary  limitations.  Thus,  they  grant  community  members  access  to  the  best  and 
brightest  human  resources  throughout  an  organization,  no  matter  what  business  unit  or  country  they  reside  in.  One 
community  builder  explained  this  benefit  when  he  stated,  “a  virtual  community  of  practice  expands  the  quality  of 
the  skill  base  that  we’re  able  to  draw  from. . . it  helps  us  get  the  best  quality  people. . . by  not  being  constrained  by 
physical  location.” 

By  granting  open  access  to  the  knowledge  and  expertise  of  a collective  whole,  VCoPs  increase  their 
members’  power  and  ability  to  function  effectively  and  efficiently.  For  example,  VCoP  members  often  help 
individuals  solve  problems  that  they  couldn’t  solve  on  their  own  in  a relatively  short  period  of  time.  By  posing  a 
problem  to  VCoP  members  around  the  globe,  these  individuals  can  get  a variety  of  contextualized  solutions  to  their 
exact  problem  or  recommendations  based  on  similar  problems  that  other  VCoP  participants  have  experienced  very 
quickly.  One  VCoP  builder  emphasized  the  problem-solving  benefit  of  VCoPs  in  the  following  statement: 

There  is  no  time  barrier,  no  geographic  barrier,  no  culture  barrier  keeping  you  from  solving  your  problem. ...  If  you 
are  in  the  middle  of  Kalimantan,  or  in  the  middle  of  the  Borneo  Islands,  you  have  the  company’s  [virtual  community 
members]  and  with  that,  you  have  the  support  of  everywhere  in  the  world  helping  you  resolve  your  problem. 

VCoPs  Increase  Communication  between  Employees  and  Management 

Another  way  that  VCoPs  boost  an  organization’s  data,  information,  and  knowledge  flow  is  increasing  the 
communication  between  management  and  employees.  VCoPs  do  this  by  providing  a means  for  employees  to  safely 
voice  opinions  and  concerns  and  introducing  a channel  through  which  management  can  solicit  individual  employee 
input  and  feedback.  The  exchange  both  directions  helps  to  break  down  the  hierarchies  that  separate  management 
from  front  line  employees.  For  example,  many  community  builders  indicated  that  individual  VCoP  members  who 
hesitate  to  voice  their  opinion,  share  their  ideas,  and  help  create  new  processes  on  their  own  will  engage  in  these 
activities  within  their  virtual  communities.  The  results  of  these  exchanges  can  be  shared  directly  with  members  of 
the  company’s  management  if  they  are  VCoP  members.  If  not,  the  VCoP  members  are  able  to  discuss  the  issue(s) 
raised  among  themselves,  and  then  pass  their  collective,  and  usually  improved  opinions,  ideas,  and 
recommendations  on  to  company  management  from  the  collective  community  membership  instead  of  a single 
individual.  In  that  situation,  VCoP  members’  feedback  and  recommendations  often  have  more  credence  with 
management  because  it  is  collective,  rather  than  feedback  from  a single  individual. 

In  the  same  manner  that  VCoPs  provide  members  with  a safe  vehicle  to  communicate  with  management, 
VCoPs  provide  managers  with  a non-invasive  mechanism  to  inform  employees  of  new  developments,  influence 
policy  and  solicit  feedback.  In  her  interview,  one  VCoP  builder  indicated  that  her  management  actively  uses  VCoPs 
to  gather  input  from  employees  in  remote  locations,  especially  when  they  are  building  new  programs  that  will 
impact  those  employees.  She  said  that  the  VCoPs  provide  an  easy  way  to  get  feedback  on  new  programs  or  policies 
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from  people  across  their  organization  instead  of  only  receiving  input  from  employees  in  the  corporate  office  as  they 
did  in  the  past.  In  addition  to  facilitating  the  creation  of  better  policies  and  programs,  VCoPs  inadvertently  increase 
employee  buy-because  they  have  opportunities  to  voice  their  opinion  about  those  policies  and  programs. 
Additionally,  virtual  communities  of  practice  establish  a forum  for  company  management  and  employees  to 
collectively  establish  standards  across  geographic  and  culture  boundaries. 

VCoPs  Extend  Communication  Between  Face-to-Face  Meetings 

VCoPs  also  increase  the  information  flow  between  employees  by  extending  communication  that  occurs  in 
face-to-face  meetings.  Members  of  VCoPs  who  meet  physically  on  a regular  basis  often  use  the  virtual  component 
of  their  communities  to  interact  with  one  another  between  these  meetings.  As  an  example,  one  VCoP  leader  said  that 
the  virtual  communication  channel  between  members  of  his  virtual  community  allows  them  to  follow  up  on  issues 
and  stay  abreast  of  concerns  facing  their  organization,  divisions,  or  product  lines  when  they  are  not  physically 
present.  It  also  gives  community  members  the  chance  to  jointly  work  on  unresolved  issues  and  action  items  assigned 
during  face-to-face  meetings  before  the  next  meeting  takes  place.  For  example,  one  VCoP  builder  commented  that 
the  addition  of  a virtual  component  to  a co-located  community  of  practice  greatly  increased  communication  between 
community  members  because  they  were  able  to  discuss  items  and  follow  up  on  tasks  that  they  had  previously 
forgotten  between  face-to-face  meetings.  This  made  both  the  face-to-face  meetings  and  the  period  of  time  between 
those  meetings  more  productive. 

VCoPs  Codify  Best  Practices  and  Solutions  to  Problems 

Another  positive  VCoP  outcome  related  to  knowledge  management  is  the  manner  in  which  the  information 
is  captured.  Since  most  of  the  communication  and  information  exchange  originating  in  virtual  communities  of 
practice  occurs  electronically,  the  questions,  solutions,  and  best  practices  exchanged  can  be  captured,  organized,  and 
archived  for  reference  at  a later  date,  in  addition  to  being  shared  among  community  members  immediately.  This 
practice  positively  impacts  an  organization’s  knowledge  management  initiative  from  both  the  codification  and 
personalization  knowledge  management  perspectives  introduced  at  the  beginning  of  this  paper.  Through  VCoPs, 
information  is  shared  between  people  in  relation  to  a specific  context;  thus  it  is  personalized.  Additionally,  because 
it  is  digitally  captured,  it  is  also  codified  to  that  others  can  benefit  from  the  exchange  in  the  future. 

One  example  of  such  a beneficial  exchange  occurred  in  a VCoP  hosted  by  a multi-national  insurance 
provider.  In  this  instance  when  individuals  in  a Canadian  office  said,  “Boy  we’ve  got  a problem  with. . .”  members 
of  the  VCoP  were  able  to  say,  “Oh,  the  Midwest  office  has  already  figured  it  out  and  they’ve  already  got  the  solution 
implemented.  Talk  to  them  and  look  at  the  information  in  the  community  archives  to  see  what  they  did.”  In  that 
single  instance,  the  VCoP  provided  both  the  personal  contacts  and  the  repository  of  information  that  saved  the 
Canadian  office  countless  hours  of  duplicate  effort  and  a large  amount  of  money. 

VCoPs  Facilitate  Informal  and  Formal  Training 

Due  to  the  nature  of  the  communication  that  occurs  in  VCoPs,  these  communities  regularly  facilitate 
informal  training.  In  many  ways,  the  questions  that  VCoP  participants  pose  to  one  another  and  their  calls  for 
problem  solving  assistance  serve  as  informal  training  requests  because  one  VCoP  member  is  seeking  information 
from  peers  or  experts  to  perform  his  or  her  job  better.  If  community  interaction  is  viewed  in  this  light,  whenever 
VCoP  members  answer  one  another’s  questions  or  provide  advice,  they  are  filling  those  informal  training  requests 
with  impromptu  training  experiences.  As  a result,  virtual  communities  of  practice  act  as  informal  training  networks 
and  they  provide  constant  informal  training  and  mentoring  opportunities.  VCoP  leaders  indicated  that  community 
members  also  impact  and  influence  formal  training  opportunities  by  identifying  where  collective  knowledge  gaps 
exist  and  requesting  formal  training  for  VCoP  members  in  order  to  fill  those  gaps. 

VCoPs  Foster  Innovation 

Virtual  communities  of  practice  also  benefit  their  sponsoring  organizations  by  fostering  innovation.  VCoP 
builders  and  leaders  indicate  that  VCoPs  tend  to  foster  innovation  and  refine  organizational  processes  because  the 
members  examine  problems  and  processes  from  multiple  perspectives  in  a non- threatening,  non-hierarchical,  non- 
constrained  environment  and  often  create  new  processes  or  streamline  existing  processes.  One  VCoP  expert  focused 
in  on  this  organizational  benefit  in  her  interview  when  she  said,  “If  you  bring  the  right  people  together  and  help 
them  share  their  tasks  and  knowledge  and  collaborate  on  problem  solving,  you  have  innovation. . . so  a community 
of  practice  is  a perfect  structure  to  engender  innovation.”  Other  VCoP  builders  echoed  this  expert’s  assertion  when 
they  told  us  that  their  VCoP  members  generally  get  excited  about  having  opportunities  to  innovate  and  take 
advantage  of  those  opportunities. 
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VCoPs  Help  Organizations  Reduce  Costs 

Virtual  communities  of  practice  can  help  companies  reduce  their  bottom  line  in  a variety  of  ways.  VCoP 
builders  and  leaders  indicated  that  by  sponsoring  virtual  communities  of  practice  they  were  able  to  reduce  the  need 
for  travel,  decrease  their  overall  training  budget,  reduce  duplicated  efforts,  and  minimize  the  time  it  takes  to 
communicate  with  VCoP  members  across  the  world.  In  particular,  several  community  builders  noted  that  by 
utilizing  virtual  communities  of  practice  as  a training  and  collaboration  tool,  they  have  been  able  to  significantly 
decrease  the  overall  training  expenses  for  their  organization.  For  example,  one  VCoP  builder  of  a worldwide  oil 
product  company  indicated  that  his  organization  saves  $35,000-$40,000  every  month  because  members  of  a specific 
community  of  practice  discuss  and  work  on  issues  that  they  used  to  discuss  in  a monthly  face-to-face  meeting 
through  the  VCoP.  This  builder  said  that  even  though  VCoPs  require  specific  software  and  other  technology  to 
communicate,  the  costs  associated  with  that  technology  are  substantially  less  than  paying  for  all  the  community 
members  to  meet  physically  on  a regular  basis  so  he  happily  provides  the  needed  software,  hardware,  and  server 
space  needed  by  the  VCoP  to  operate  effectively. 

Several  community  builders  also  noted  that  sponsoring  virtual  communities  can  help  organizations  reduce 
costs  incurred  by  duplicating  efforts.  One  VCoP  builder  stated  that  VCoPs  “help  out  the  bottom  line”  as  they  start  to 
reduce  redundancies  and  eliminate  duplicated  efforts.  Another  VCoP  leader  quantified  the  costs  savings  achieved  by 
a particular  community  in  terms  of  his  personal  time  savings.  He  commented  that  he  is  able  to  achieve  the  same 
gains  by  relaying  a message  to  members  of  this  community  virtually  in  a couple  of  hours  as  he  did  when  he  traveled 
for  two  or  three  days  to  relay  the  same  message  with  community  members  physically. 

Conclusion 

The  research  data  indicate  that  virtual  communities  of  practice  benefit  the  knowledge  management 
initiatives  of  the  organizations  that  sponsor  them.  The  interviewees’  comments  evidence  the  fact  that  the  interaction 
between  VCoP  members  increases  the  amount  of  data,  information,  and  knowledge  that  is  exchanged  throughout  an 
organization  and  heightens  employees’  awareness  of  what  others  are  doing.  The  information  exchange  mutually 
benefits  community  members  throughout  organizations  and  the  organizations  themselves.  The  data  also  indicate  that 
VCoPs  increase  opportunities  for  training  and  innovation,  and  help  to  reduce  costs.  These  combined  benefits  lead  to 
improved  management  of  an  organization’s  knowledge  because  they  facilitate  knowledge  generation,  knowledge 
codification  and  coordination,  and  knowledge  transfer  throughout  organizations  (Davenport  & Prusak,  2000). 

This  study’ s findings  contribute  to  the  growing  body  of  knowledge  about  virtual  communities  of  practice 
by  identifying  specific  ways  that  organizations  benefit  from  sponsoring  VCoPs  and  by  revealing  critical  factors  that 
organizations  can  influence  to  strengthen  the  likelihood  that  these  communities  will  succeed.  As  a result,  the  data  are 
valuable  for  organizations  who  desire  to  strengthen  their  ability  to  manage  their  collective  employees’  knowledge 
and  utilize  virtual  communities  of  practice  in  that  endeavor.  The  VCoP  builders  and  leaders  who  participated  in  the 
study  collectively  articulated  that  greater  information  and  communication  flow,  increased  opportunities  for  training 
and  innovation,  better  utilization  of  global  resources,  and  reduced  costs  are  benefits  that  organizations  can  realize 
from  supporting  virtual  communities  of  practice.  These  benefits  directly  lead  to  improved  knowledge  management 
on  both  the  individual  and  organizational  level  because  the  virtual  communities  of  practice  serve  as  organizational 
networks  where  knowledge  is  created,  disseminated,  and  transferred  throughout  organizations. 
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Introduction 

The  existence  of  a "digital  divide"  in  which  portions  of  society  do  not  have  sufficient  access  to  technology, 
nor  to  the  information  and  skills  that  technology  use  imparts,  was  of  concern  for  educators  and  policy  makers  even 
before  home  computers  were  easily  connected  to  the  Internet  (NTIA  1995,  1998,  1999,  2000;  Resmer,  Mingle  & 
Oblinger,  1995;  Riley,  1996;  Annie  E.  Casey  Foundation,  1998).  As  new  technologies  (such  as  broadband  Internet 
access),  and  new  uses  for  technology  (such  as  interactive  websites  for  homework)  are  developed,  new  levels  of  the 
divide  will  appear,  certainly  to  the  extent  that  children  in  low  income  homes  do  not  gain  access  to  the  same  level  of 
information  at  the  same  rate  as  other  children  (c.f.  Hundt,  2003;  Digital  Divide  Network,  2003).  In  an  era  in  which 
funding  may  be  difficult  to  obtain,  the  development  of  tested,  scalable,  affordable  solutions  should  therefore  be  a 
mandate  for  educators. 

Research  in  the  digital  divide  field  is  still  in  the  critical  early  stages;  consequently,  much  of  the  research  has 
focused  on  demographic  information  and  on  documentation  of  participant  benefits  (cf.  Chow,  Ellis,  Walker  & Wise, 
2000).  As  it  is  now  well  established  that  participants  do  benefit,  and  that  those  who  would  not  otherwise  have  access 
to  technology  do  take  advantage  of  such  opportunities,  more  focused  research  is  called  for.  In  addition,  much  early 
digital  divide  research  has  been  based  on  solutions  developed  in  conjunction  with  community  technology  centers 
(Chow,  Ellis,  Walker  & Wise,  2000);  the  notable  exceptions  (e.g.  Apple  Classroom  of  Tomorrow,  1995;  Pinkett, 
2000,  2002)  were  largely  based  on  partnerships  with  specific  institutions  and  on  highly  funded  pilot  programs,  so 
that  conclusions  drawn  may  be  limited  in  the  extent  to  which  they  can  be  used  to  plan  larger  programs  and  fiscally 
reasonable  solutions.  Moreover,  as  Secretary  of  Education  Rod  Paige  has  pointed  out  (US  Dept,  of  Ed.,  2003a, 
2003b,  2003c),  access  in  schools  and  in  other  institutions  such  as  libraries  does  not  substitute  for  access  to  home 
computers  with  Internet  connectivity.  Projects  designed  to  test  home-based  technology  programs  are  thus  seen  to  be 
apriority  (Andrews,  DiGangi,  & Jannasch-Pennell,  1999,  2000;  Stock,  2001). 

This  paper  reports  on  a seven-year  project  that  was  specifically  designed  to  generate  scalable  and 
affordable  solutions  to  the  digital  divide  in  a participatory  research  setting,  and  on  a briefer  study  based  on  the 
project;  one  year  was  allocated  to  data  collection  in  the  study,  and  a second  year  was  devoted  to  completing  the 
analysis.  The  paper  offers  an  overview  of  results  of  the  study,  and  of  the  means  by  which  the  results  were  obtained. 
While  more  detailed  papers  are  planned,  it  is  hoped  that  this  overview  will  draw  attention  not  only  to  the  divide,  but 
also  to  some  solutions,  and  to  the  efficacy  of  a grassroots-oriented  means  of  generating  such  solutions. 

In  the  seven-year  project,  computers  were  placed  in  low  income  homes  as  well  as  in  non-standard  dwelling  places 
(e.g.  shelters  and  vehicles)  via  a participant-led  grassroots  technology  program,  Floaters.org.  Outcomes  regarding 
disposition  and  use  of  the  computers  were  tracked  largely  via  a peer  mentoring  process  in  which  previous  recipients 
introduced  new  members  to  the  computers.  Mentors  in  particular,  and  mentees  as  well,  were  invited  to  bring  their 
experiences  and  suggestions  to  a participant  research  group  and  in  so  doing  to  serve  as  a bridge  to  trained 
researchers.  The  participant  research  group  met  weekly  and  functioned  as  a consensus-based  decision-making  body 
with  regard  to  the  evolution  of  the  project.  Participants  thus  shaped  the  program  to  their  needs  in  an  exemplary 
participatory  action  research  project  (c.f.  Moser,  1977;  Fals  Borda,  1995;  Sohng,  1996). 

Five  years  into  the  project,  a mixed-method  study  was  begun  that  would  focus  on  specific  aspects  of  home- 
based  technology  integration.  Methods  were  chosen,  with  participant  input,  to  maximize  the  amount  of  information 
gained.  Thus,  discourse  analysis  was  selected  as  the  appropriate  method  to  examine  transcripts  of  taped  peer 
mentoring  sessions;  grounded  theory  was  used  to  identify  changes  to  the  program  that  emerged  as  the  program  was 
scaled  up  in  size;  and  a standard  self-efficacy  scale  was  used  to  examine  participant  self  efficacy  with  regard  to 
technology  occupations.  Finally,  the  program  looked  back  at  itself  via  the  process  of  participatory  action  research, 
used  in  the  project  in  order  to  generate  and  evaluate  digital  divide  solutions  from  the  bottom  up,  that  is,  with 
maximum  input  from  the  people  most  affected. 
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Purposes  of  the  Project  and  Study,  and  Research  Questions  for  the  Study 

Participant-chosen  goals  for  the  project,  and  therefore  for  the  participatory  aspect  of  the  study  as  well,  were 
to  investigate  how  best  technology  could  be  integrated  into  the  homes  and  lives  of  low  income  populations;  and, 
consequently,  how  technology  could  be  used  to  improve  these  lives.  From  the  beginning,  the  project  sought  not  only 
demographic  information  and  information  about  participant  benefits,  but  also  information  about  how  those  benefits 
could  be  accomplished. 

Goals  of  the  study,  as  contrasted  with  those  of  the  project,  were  chosen  in  participant-driven  focus  groups 
in  which  members  learned  research  methods.  These  goals  focused  specifically  on  a set  of  particular  research 
questions: 

What  took  place  with  regard  to  teaching  and  learning  in  the  peer  mentoring  sessions? 

How  did  the  structure  of  the  program,  designed  over  time  by  participants,  change  further  as  the  program 
was  scaled  up  in  size? 

Did  participant  self  efficacy  change  during  the  study  with  regard  to  occupations,  especially  with  regard  to 
technology  occupations? 


About  the  Floaters.org  Project 

Working  with  university  researchers,  a community  group  designed  and  implemented  the  Floaters.org 
project,  in  which  older  computers  were  and  are  recycled  into  low-income  homes  and  then  monitored.  The 
Floaters.org  project  itself  is  a participatory  action  research  project,  wherein  all  participants  have  an  equal  voice  and 
goals  are  set  by  group  members  working  together.  The  project  was  designed  to  integrate  technology  with  those  who 
are  least  likely  to  otherwise  attain  it:  specifically,  those  living  in  poverty,  those  who  have  been  homeless  and  later, 
people  with  disabilities  who  could  not  otherwise  afford  technology.  The  project  simultaneously  was  designed  to 
identify  and  put  into  practice  the  highest  pedagogical  and  research  practices.  This  does  not  imply  that  mistakes  were 
not  made,  but  rather  that  learning  from  mistakes  was  a built-in  part  of  the  process. 

An  examination  of  the  nature  of  the  project,  undertaken  as  the  two  year  study  began,  showed  that 
participants  took  on  three  roles:  peer  mentors  or  mentees;  shapers  of  the  program;  and  co-researchers.  Each  role 
corresponds  to  one  of  the  research  questions  stated  above. 

The  monitored  study  was  designed  with  input  from  participants,  and  participant  approval  was  gained  for  all 
aspects  of  the  study.  As  part  of  their  work  as  co-researchers,  participants  learned  about  ethics  standards,  including 
the  right  to  withdraw  from  the  study  without  having  to  withdraw  from  the  program.  Each  participant  chose  a screen 
name  or  alias,  and  data  were  recorded  for  these  aliases.  Where  privacy  dictated,  the  academic  researcher 
occasionally  chose  to  use  an  additional  naming  convention  as  well.  Those  participants  who  did  leave  the  study  have 
given  their  permission  to  continue  using  their  data. 

Discourse  Analysis 

Discourse  analysis,  a qualitative  research  method  used  to  examine  the  peer  mentoring  sessions,  consists  of 
a number  of  approaches  to  analysis  of  verbal  interactions.  Discourse  analysis  was  agreed  upon  as  a method  for  the 
study  by  the  participants,  who  saw  this  method  as  a means  of  investigating  empowerment.  This  view  is  consistent 
with  the  literature:  in  Language  and  Power,  Fairclough  (1989)  invited  both  researchers  and  lay  people  to  investigate 
the  uses  of  discourse  strategies  to  take  and  consolidate  power. 

It  is  the  branch  of  discourse  analysis  known  as  conversation  analysis,  or  CA,  that  was  primarily  used  in  this 
study.  In  conversation  analysis,  audio  or  video  recordings  are  transcribed,  and  the  transcriptions  then  analyzed,  with 
structural  details  taking  precedence  over  content.  That  is,  rather  than  interpreting  data  and  assigning  motives, 
conversation  analysts  seek  information  about  the  structure  of  the  conversation.  Intonations,  pauses,  interruptions  and 
grammatical  structures  are  among  the  features  typically  coded  for. 

Conversation  analysis  was  pioneered  by  sociologist  Harvey  Sacks  in  the  1960’s;  his  work,  edited  by  Gail 
Jefferson  and  Emmanuel  Schegloff,  was  not  published  until  1992,  as  the  Lectures  on  Conversation  (Sacks,  1992). 
However,  long  before  this  publication  date,  Sacks,  Jefferson,  and  Schegloff  had  began  a collaboration  that  resulted 
in  a series  of  works  that  defined  conversation  analysis.  These  works  included  the  Semiotica  article  "Opening  up 
Closings"  in  which  Schegloff  and  Sacks  proposed  the  concept  of  the  adjacency  pair  to  explain  conversational 
structures  such  as  questions  and  answers  (Schegloff  & Sacks,  1973)  as  well  as  the  Language  article,  "A  Simplest 
Systematics  for  the  Organization  of  Turn-Taking  for  Conversation"  (Sacks,  Schegloff  and  Jefferson,  1974).  Working 
with  Sacks'  material,  Gail  Jefferson  originated  a set  of  notations  for  marking  up  text  that  was  published  with  the 
1974  article,  and  that  has  remained  standard  among  conversation  analysts. 
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In  addition  to  Sacks'  pioneering  work,  Sacks,  Schlegloff  and  Jefferson  built  on  the  work  of  Harold 
Garfinkel,  founder  of  the  ethnomethodology  school  of  sociology.  Ethnomethodology  focuses  on  the  study  of 
everyday  actions,  including  language  use,  and  takes  for  granted  that  ordinary  people  understand  what  they  are  doing; 
thus  it  is  by  nature  well  suited  to  a participatory  action  research  project  in  which  participants  are  co-researchers. 
Garfinkel  presented  the  principles  of  his  approach  in  Studies  in  Ethnomethodology,  citing  transcripts  of 
conversations  to  show  how  people  organize  their  shared  realities  using  "common  understandings"  (Garfinkel,  1967, 
p.  38).  In  other  words,  ordinary  people  can  and  do  understand  their  own  actions.  Like  the  conversation  analysts, 
Garfinkel  relied  on  observable  data,  using  only  as  much  context  as  was  necessary  to  understand  the  data  and 
expressing  a belief  that  only  those  involved  could  understand  the  whole  situation  (Garfinkel,  1967). 

Also  related  is  the  work  of  Erving  Goffmann.  In  The  Presentation  of  Self  in  Everyday  Life,  Goffman 
showed  how,  whether  consciously  or  not,  conversationalists  act  out  roles  and  create  a "front"-"that  part  of  the 
individual's  performance  which  regularly  functions  in  a general  and  fixed  fashion  to  define  the  situation  for  those 
who  observe  the  performance"  (Goffman,  1959,  p.22).  Conversation  analysis  draws  from  the  work  of  Goffmann  in 
that  both  reveal  the  ways  in  which  relationships  are  constructed  and  roles  created. 

Transcription  in  itself  has  come  to  be  viewed  as  a theoretical  process,  a viewpoint  introduced  by  Elinor 
Ochs  in  "Transcription  as  Theory"  (1979)  and  by  Carole  Edelsky,  in  "Who's  Got  the  Floor?"  (1981).  Central  here  are 
the  realizations  that  the  raw  data  are  not  the  same  as  the  transcribed  data;  that,  rather,  the  transcription  is  a 
representation  of  the  data;  and  that  the  infinite  multitude  of  decisions  made  during  transcription  are  actually 
theoretical  decisions  reflecting  the  relative  importance  of  particular  discourse  features.  Two  researchers  working 
independently  would  not  make  the  same  theoretical  decisions;  and  a single  researcher,  writing  about  the  same  text  at 
different  times,  may  make  different  coding  decisions. 

It  cannot,  therefore,  be  assumed  that  reliance  on  observable  data  over  inference  in  conversation  analysis 
implies  that  the  data  are  wholly  objective.  The  ongoing  development  of  software  applications  that  can  be  brought  to 
bear  on  the  transcription  process  (for  example,  software  that  measures  pauses)  does  not  change  this,  as  researchers 
will  continue  to  make  the  important  decisions:  for  example,  pauses  in  a technology  mentoring  session  may  have  to 
do  not  with  hesitation  but  with  mentee  concentration  on  a technology  task.  Where  there  is  reason,  a second 
researcher  may  separately  transcribe  audio  or  video  recordings,  and  the  two  versions  can  be  compared;  or,  a group 
of  researchers  working  together  may  listen/view  and  transcribe  as  a group,  as  is  recommended  procedure  in 
ethnographic  inquiry  and  interaction  analysis  (Jordan  & Henderson,  1995).  However,  neither  process  is  necessarily 
indicated  if  it  is  accepted  that  transcription  decisions  are  properly  the  domain  of  the  researcher  (c.f.  Ochs,  1979; 
Edelsky,  1981;  Lemke,  1998). 

A second  branch  of  discourse  analysis  useful  in  a participatory  action  research  study  is  critical  discourse 
analysis,  or  CDA,  in  which  discourse  is  examined  for  underlying  relationships  of  power.  In  the  European  countries, 
Foucault  is  thought  of  as  the  father  of  discourse  analysis  (e.g.  Fricke,  1999;  Diaz-Bone,  2003),  in  part  for  work 
published  in  his  Archeology  of  Knowledge  and  in  his  Discourse  on  Language,  published  together  in  English  in 
1972.  Elsewhere,  Foucault  might  be  thought  of  at  least  as  the  godfather  of  critical  discourse  analysis:  Foucault  began 
the  Archeology  of  Knowledge  by  declaring  that  in  order  to  understand  any  discourse  it  would  be  necessary  first  to 
get  rid  of  all  preconceived  ideas  in  order  to  begin  anew,  and  then  to  examine  relations,  including  relations  of  power. 

The  Floaters.org  study  is  closer  in  its  approach  to  another  critical  discourse  analyst,  Norman  Fairclough. 
Fairclough's  use  of  close  readings  of  text  resembles  that  of  conversation  analysis,  although  his  intention  is  to  identify 
and  uncover  power  relations.  In  Language  and  Power  (1989),  Fairclough  not  only  identified  discourse  structures  that 
give  and  take  power,  but  invited  others  (including  those  with  no  prior  academic  background)  to  do  so  as  well. 

Participatory  Action  Research 

The  term  “action  research”  has  several  meanings.  In  educational  research,  action  research  might  be 
classified  into  two  major  branches,  practitioner  research  and  participatory  action  research  or  PAR.  In  both,  research 
is  carried  on  at  least  in  part  by  people  who  have  not  been  formally  trained  as  researchers,  but  who  have  the 
advantage  of  being  directly  within  the  research  situation  and  so  have  access  to  an  unusually  high  level  of 
information.  Practitioner  research  in  education  rests  on  the  belief  that  teachers  (“practitioners”)  can  participate  in 
research,  generally  via  self  study,  rather  than  being  consumers  of  research  only.  In  “participatory  action  research”  on 
the  other  hand,  the  object  is  goal-oriented  social  justice,  as  guided  by  those  affected.  In  PAR,  a researcher  or 
researchers  may  enter  the  situation  with  the  intent  of  forming  a participatory  research  group.  The  Floaters.org  study 
utilizes  the  PAR  approach:  the  initial  impulse  came  from  a researcher,  who  invited  a counterpart  in  an  affected 
community  to  join  together  in  investigating  technology  integration  by  distributing  donated  computers. 

Further  confusion  as  to  the  nature  of  action  research  arises  because  participatory  investigation  or 
participatory  action  research  has  been  used  widely  by  such  entities  as  the  World  Bank  (Narayan,  Patel,  Schafft, 
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Rademacher  & Koch-Schulte,  2000).  Such  entities  generally  have  an  a priori  interest  in  showing  immediate  financial 
gains  for  participants,  whereas  true  participatory  action  research  centers  on  the  identification  of  participant  values 
prior  to  the  taking  of  action  (Fals  Borda,  1995;  Sohng,  1996). 

Participatory  action  research  has  been  used  in  the  United  States  in  the  establishment  of  support  groups  for 
sufferers  from  AIDS  (Glasser  & Bridgman,  1999)  and  in  the  investigation  of  the  efficacy  of  prison  education.  (Fine, 
Boudin,  Bowen,  Clark,  Hylton,  Migdalia,  Missy,  Rivera,  Robers,  Smart,  Torre  & Upegui,  2001).  In  education,  the 
work  of  Paolo  Freire  can  also  be  seen  as  participatory  action  research,  in  that  Freire  mobilized  participants  to  take 
control  of  their  education  (Freire,  1970,  1994,  1998). 

In  participatory  action  research,  participants  are  co-researchers,  with  an  equal  voice  in  all  decisions.  The 
method  itself  therefore  is  also  open  to  further  refinement.  Before  the  Floaters.org  study  began,  five  years’  of  group 
work  on  the  project  had  resulted  in  three  principles  against  which  all  decisions  were  tested:  equality  of  voice; 
consensus-based  decision  making;  and  revolving  authority,  in  which  those  who  are  expert  in  a particular  area  may 
step  forward  and  take  charge,  so  long  as  those  who  are  interested  in  the  area  may  also  participate. 

In  interactions  outside  of  the  group,  “Nothing  about  us  without  us”  is  the  credo  of  grassroots  participatory 
action  research,  as  documented  on  the  Homeless  People’s  Network,  the  sister  website  and  mailing  list  for  the 
Floaters.org  project  (c.f.  also  Charlton,  2000).  “Speak  truth  to  power,”  is  a related  imperative,  implying  that 
participants  must  at  every  step  of  the  way  open  to  stating  their  opinion  rather  than,  for  example,  avoiding  the 
situation  (Carey  et  al,  1955;  Kennedy  Cuomo  & Adams,  2004). 

With  these  guiding  principles  in  participatory  action  research,  project  members  take  an  ethical  stance  that  is 
every  bit  as  difficult  to  implement  as  the  most  rigorous  of  quantitative  methods;  in  both,  at  the  end  of  the  study,  the 
researchers  know  that  their  findings  are  trustworthy  to  the  extent  that  the  method  has  been  properly  followed. 

Self  Efficacy 

"Perceived  self  efficacy"  refers  to  people's  optimistic  beliefs  about  their  ability  to  reach  their  goals 
(Bandura,  1977,  2001).  Hackett  and  Betz  have  stated  that  one  of  the  most  useful  concepts  in  modern  psychology  is 
Albert  Bandura's  concept  of  self-efficacy  expectations,  for  self  efficacy  has  been  shown  to  affect  human 
development  (Hackett  & Betz,  1981;  Betz  & Hackett,  1981;  Bandura,  1977;  Bandura,  2001).  Changes  in  self 
efficacy  have  four  sources,  according  to  Bandura:  mastery  experiences,  vicarious  experiences  modeled  by  others, 
verbal  persuasion,  and  people's  own  physiological  indicators  or  somatic  and  emotional  states  resulting  from  the 
attempt  to  achieve.  Investigating  these  four  sources,  Bandura  found,  for  example,  that  verbal  dissuasion  is  easier  to 
accomplish  than  verbal  persuasion;  that  both  positive  and  negative  moods  may  affect  self  efficacy,  as  can  pain;  and 
that,  in  a learning  environment,  teacher  self  efficacy  can  negatively  or  positively  impact  student  self  efficacy 
(Bandura,  1994,  1995,  1997,  2001).  Bandura’s  work  with  self  efficacy  was  a part  of  his  pioneering  work  that  made  a 
place  for  cognition,  rather  than  adhering  to  the  older  “black  box”  theories  of  psychology.  In  thus  allowing  for  the 
thought  processes  of  the  individual,  Bandura  created  a concept  which  is  more  appropriate  to  grassroots  PAR 
investigation,  which  also  places  emphasis  on  the  individual  participant,  than  earlier  theories. 

Self  efficacy  is  domain  specific.  Bandura  states  in  his  Guide  to  the  Construction  of  Self-Efficacy  Scales 
(2001)  that  is  important  to  use  a domain- specific  measure  rather  than  a general  measure  of  efficacy,  as  general 
measures  are  too  ambiguous  to  be  meaningful.  Self  efficacy  can  transfer  across  domains,  and  powerful  experiences, 
in  particular,  can  effect  changes  across  many  domains  (Bandura,  1994,  1995,  1997,  2001). 

With  regard  to  the  study  of  self  efficacy  with  low  income  populations,  early  research  conducted  by  CTCNet 
(CTCNet,  acc.  2003)  identified  a number  of  benefits  to  users,  one  of  which  was  personal  efficacy  (Mark,  Cornebise 
& Wahl,  1997).  In  subsequent  CTCNet  studies,  this  benefit  was  dropped  as  a concept  of  interest  (Chow  et.  al., 

2000),  but  a more  domain- specific  self-efficacy  concept  may  be  worth  a closer  examination. 

While  career  goals  are  not  the  only  objective  of  technology  integration,  they  are  a concept  of  interest 
among  the  Floaters.org  population,  as  determined  by  the  participatory  research  group.  "Work  self  efficacy"  or 
"Occupational  self  efficacy"  refers  to  the  belief  that  one  can  succeed  in  a particular  job.  The  concept  of  occupational 
self  efficacy  itself  was  introduced  in  pioneering  work  published  in  1981  by  Gail  Hackett  and  Nancy  Betz  (Hackett  & 
Betz,  1981;  Betz  & Hackett,  1981).  In  these  works,  Betz  and  Hackett  applied  the  concept  of  self  efficacy  to  career 
counseling  and  discovered  gender  differences  among  with  regard  to  traditionally  gendered  occupations,  in  particular, 
men  had  much  stronger  self  efficacy  scores  for  traditionally  male  careers  (Betz  & Hackett,  1981;  Hackett  & Betz, 
1981.) 

Following  Bandura's  model,  Hackett  and  Betz  created  the  Occupational  Self-Efficacy  Scale  (OSES)  in  the 
form  of  a Likert  scale  from  one  to  ten  for  20  occupations;  respondents  were  asked  to  select  their  answers  based  first 
on  how  certain  they  were  of  being  able  to  complete  the  education  or  training  for  each  occupation,  and  then  on  how 
certain  they  were  of  being  able  to  carry  out  the  job  duties  (Betz  & Hackett,  1998).  A revised  version  of  their  1981 
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Manual  for  the  Occupational  Self-Efficacy  Scale  was  made  available  online  in  1998  (Betz  & Hackett,  1998). 
Typically  the  scale  is  revised  by  the  researcher  to  offer  relevant  occupations  for  the  population  of  interest  (Hackett, 
personal  communications,  2001,  2003);  in  this  PAR  study,  the  participants  themselves  chose  the  relevant 
occupations.  The  scale  can  be  administered  in  terms  of  the  training  aspect,  in  terms  of  the  job  duties  aspect,  or  with 
both  aspects  included. 


Method 

Participants 

Participants  in  the  group  (n=184  at  the  time  of  the  study)  comprise  a “snowball”  sample  (Babbie,  1998)  in 
that  new  participants  were  recruited  by  project  members,  each  of  whom  was  asked  to  choose  someone  else  to  teach. 
For  the  study,  37  primary  participants  were  lent  recycled  computers,  standardized  to  the  extent  possible.  Data  were 
collected  over  the  course  of  a year  for  each  of  the  37.  Each  primary  participant  represented  a different  family  except 
where  there  were  two  or  more  computers  in  a family;  in  these  cases,  each  primary  user  of  a computer  was  also  a 
primary  participant  in  the  study.  All  of  the  37  volunteered  for  the  project,  and  all  were  over  the  age  of  eighteen. 

Each  section  of  the  study  drew  on  a different  subset  of  participants  as  appropriate: 

Fourteen  of  the  37  took  part  in  the  taped  peer  mentoring  sessions.  The  taped  sessions  used  for  the  study 
were  theoretically  sampled,  that  is,  they  were  chosen  in  order  to  provide  as  wide  a range  as  possible  in  terms  of  age, 
level  of  experience  in  the  program,  gender,  and  level  of  education. 

All  participants  were  invited  to  optional  weekly  research  meetings,  where  data  were  collected  in  the  form 
of  field  notes  and  observations  regarding  changes  in  the  structure  of  the  Floaters.org  project  during  the  study. 

Twenty-two  members  agreed  to  take  part  in  the  self  efficacy  portion  of  the  study;  of  these,  seventeen  took 
part  in  both  administrations  of  the  self-efficacy  scale,  at  approximately  four-month  intervals,  while  five  took  part 
only  in  the  first  administration.  The  22  were  self-selected  from  the  37  primary  participants. 

While  research  decisions  (e.g.  aspects  studied  and  methods)  were  made  by  the  participants  after  group 
study  of  research  methods,  and  while  participants  took  part  in  other  aspects  of  the  research  as  appropriate  (e.g., 
participants  selected  the  careers  to  appear  on  the  Occupational  Self  Efficacy  Scale),  a trained  researcher  conducted 
the  analysis,  with  constant  member  checks  wherein  participants  were  asked  to  comment  on  portions  of  the  analysis. 

Data  Sources 

Data  sources  included  the  following: 

• Records  of  the  computers  and  the  work  done  with  them 

• Field  notes,  observations,  and  qualitative  memos 

• Eight  videotaped  mentoring  sessions,  theoretically  sampled  so  as  to  provide  the  widest  possible  variety  of 
mentors  and  mentees 

• Field  notes  and  observations  from  focus  group  sessions 

• Scores  from  the  two  administrations  of  the  Occupational  Self  Efficacy  Scale 
Responses  to  followup  questions  asked  of  those  who  took  part  in  the  mentoring  study 

Procedure 

Recycled  Macintosh  computers  with  a standard  set  of  applications  were  distributed  to  thirty-seven  primary 
participants,  along  with  self-study  materials  and  resources  for  feedback. 

Eight  peer  mentoring  sessions,  theoretically  sampled  to  provide  the  widest  possible  range  in  education, 
experience,  age,  and  gender,  were  videotaped  by  the  researcher. 

The  Occupational  Self-Efficacy  Scale  was  administered  at  approximately  four-month  intervals  to  a subset 
of  twenty-two  self- selected  mentors  and  mentees  in  order  to  determine  any  changes  in  attitude  towards  computer- 
related  jobs.  Seventeen  of  the  twenty-two  were  available  to  take  the  self-efficacy  instrument  a second  time;  five  took 
it  only  once. 

Weekly  focus  group  sessions/research  group  sessions  were  held,  and  follow-up  interviews  were  scheduled 
during  the  last  three  months. 
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Figure  1 . Timeline  for  the  study.  The  equivalent  of  the  recommended  year  for  ethnographic  data  collection  was 
allotted  to  data  collection  for  the  study,  along  with  a second,  also  recommended,  year  for  analysis  and  writing. 

Analysis 

Peer  Mentoring  Sessions:  A Discourse  Analysis 

The  first  of  the  three  aspects  studied  was  that  of  the  mentor/mentee  relationship.  For  the  discourse  analysis 
portion  of  the  study,  eight  theoretically  sampled  peer  mentoring  sessions  were  taped,  transcribed,  and  coded  for 
discourse  features  in  order  to  examine  the  participant  roles  of  mentor  and  mentee.  The  sessions  were  selected  via  the 
theoretical  sampling  process  in  order  to  provide  as  wide  a range  as  possible  regarding  age,  level  of  experience  in  the 
program,  gender,  and  level  of  education.  The  mentoring  pairs  were  videotaped  during  sessions,  and  the  sessions 
were  transcribed  and  marked  up,  then  coded,  using  modified  conventions  based  on  the  work  of  Gail  Jefferson 
(Sacks,  Schegloff  & Jefferson,  1974;  Jefferson,  1984).  It  is  standard  practice  for  the  researcher  to  adjust  the 
transcription  conventions  to  the  requirements  of  the  researcher’ s interests  and  of  the  text  at  hand,  so  the  conventions 
given  here  vary  from  those  originated  in  1974.  Data  were  coded  line  by  line,  in  constant  comparison  analysis 
(Glesne  & Peshkin)  until  categories  were  found  and  then  saturated,  that  is,  until  all  new  data  fit  into  already 
discovered  categories.  Transcription  conventions  used  for  marking  up  the  text  can  be  found  in  Appendix  A. 

Scalability/Changes  to  the  Program 

The  second  aspect  studied  was  that  of  scalability  of  the  project.  At  the  beginning  of  the  study,  the  project 
was  scaled  up  in  size.  Rather  than  accepting  new  members  one  at  a time  as  mentors  became  available,  twenty  new 
families  were  accepted  at  once.  As  the  program  was  modified  in  order  to  accomplish  this  expansion  of  the  program 
and  to  provide  ways  to  collect  data,  the  opportunity  arose  to  track  answering  changes  produced  by  the  participants 
responsible  for  the  program  structure.  Most  important  of  these  changes  was  the  establishment  of  a weekly  focus 
group  that  any  participant  (n=184)  could  attend  and  that  served  as  a means  of  communicating  concerns  as  well  as  to 
continue  the  work  of  shaping  the  program.  Field  notes  of  these  focus  group  meetings  were  collected  by  the 
researcher  and  analyzed,  again  via  coding  and  constant  comparative  analysis,  in  order  to  investigate  how  the  role  of 
shaper  and  developer  of  the  program  played  out  during  this  time. 

Occupational  Self  Efficacy 

To  examine  occupational  self-efficacy,  the  Occupational  Self-Efficacy  Scale  or  OSES  was  revised  in 
accordance  with  guidelines  set  by  Bandura  (2001)  and  advice  given  by  Hackett  (personal  communications,  2001, 
2003).  The  OSES  is  typically  revised  by  the  researcher,  who  selects  occupations  that  are  relevant  to  the  group  under 
study;  here,  the  participants  were  asked  to  choose  the  occupations.  Rather  than  limit  the  list  of  occupations,  they 
chose  to  use  them  all,  adding  eight  more  technology  occupations  and  the  occupation  of  President. 

For  this  aspect  of  the  study,  four  smaller  questions  were  formulated: 
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1.  Will  there  be  a significant  difference  between  means  for  technology-related  occupations  and  non-technology 
related  occupations  within  the  first  administration  of  the  instrument? 

2.  Will  there  be  a significant  difference  between  means  of  technology-related  occupations  and  non-technology 
related  occupations  within  the  second  administration  of  the  instrument? 

3.  Will  there  be  a significant  difference  between  means  of  non-technology  occupations,  between  the  two 
administrations  of  the  instrument? 

4.  Will  there  be  a significant  difference  between  means  of  technology-related  occupations,  between  the  two 
administrations  of  the  instrument? 

Scores  on  the  self-efficacy  scales  were  analyzed  for  each  of  the  four  smaller  research  questions  by  running 
a paired  samples  t-test  to  test  for  difference  of  means.  Follow-up  questions  were  generated  based  on  a pattern 
analysis  of  the  results,  and  participants  were  contacted  and  interviewed. 

Results 

Peer  Mentoring  Sessions:  A Discourse  Analysis 

Analysis  of  the  peer  mentoring  sessions  showed  that  peer  mentors  developed  and  used  sophisticated 
teaching  strategies  and  that  these  strategies  were  similar  among  experienced  mentors,  regardless  of  education  levels. 
Coding  yielded  two  sets  of  mentor  discourse  strategies  that  had  to  do  with  verbal  contributions  of  the  mentee  to  the 
session.  Differences  were  found  in  the  use  of  these  strategies  between  experienced  and  beginning  mentor  discourse: 
expert  mentors  alternated  the  two  types  of  strategies,  thereby  encouraging  mentee  contributions  while  advancing  the 
instruction.  On  the  other  hand,  features  found  in  the  discourse  of  experienced  mentors  were  similar  regardless  of 
educational  level:  members  with  and  without  a high  school  diploma  were  equally  expert  in  mentoring  if  they  had  an 
equal  amount  of  mentoring  experience. 

One  set  of  strategies  encouraged  mentee  participation:  these  strategies  included  such  features  as  questions, 
problem  statements,  and  off-topic  remarks.  This  problem  statement  by  the  mentor,  for  example,  is  followed  by  a 
mentee  response: 

-> Mentor:  30.  I’ve  never  seen  the  mouse 

3 1 . connected  to  the  cord  before. 

Mentee  32.  (.)  Then  nobody  can  steal  them. 

The  second  set  of  strategies  included  such  discourse  features  as  interrupting,  ignoring  mentee  contributions,  and 
speeding  up  the  rate  of  mentor  speech.  Here,  the  mentor  interrupts,  thereby  retaining  control  of  the  instruction: 

->  Mentee:  23.  Let  me  show  [you  ( ) 

->  Mentor:  24.  [And  then  you  can,  now  if  you  want  to  do. 

25.  ° Let  me  show  you.° 

Scalability/Changes  to  the  Program 

Participants  modified  the  Floaters.org  program  further  during  the  study,  primarily  increasing 
communication  opportunities.  The  first  modification,  in  which  participants  requested  that  the  focus  and  research 
group  meet  weekly,  and  that  it  be  immediately  opened  to  mentors  and  mentees  alike,  provided  a regular  opportunity 
for  participants  to  bring  up  their  concerns  and  to  suggest  changes.  Participants  successfully  maintained  the  basic 
premises  of  the  program  (equality  of  voice,  consensus-based  decision  making,  and  revolving  authority)  throughout 
the  mixed  method  study:  no  mean  feat,  for  as  the  group  learned,  in  a research  situation  methods  can  easily  deviate 
from  grassroots  principles  unless  there  is  constant  attention  to  the  principles.  A second  major  modification  was  that 
of  the  addition  of  a fourth  principle  to  group  interactions:  speaking  truth  to  power  within  the  group  (as  well  as 
outside  of  it)  rather  than  walking  away.  This  addition  to  the  internal  principles  of  participatory  action  research  was 
needed  since  many  participants  (as  documented  in  the  self  efficacy  interviews)  perceived  of  their  power  as  limited  in 
some  senses,  but  strong  in  that  they  stated  that  they  could  always  walk  away  from  a situation  in  which  they  were  not 
being  treated  fairly.  Encouragement  to  speak  out  in  the  group  over  perceived  slights  was  the  solution  to  losing 
participants  over  such  problems. 


Occupational  Self  Efficacy 

Results  of  the  self-efficacy  questionnaire  were  inconclusive,  though  a major  history  effect  may  have  had 
some  impact  on  these  results.  The  study  was  interrupted  by  the  events  of  September  11,  2001,  and  in  follow-up 
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interviews  as  well  as  in  focus  group  observations,  some  participants  indicated  that  these  events  had  changed  their 
opinions. 

One  significant  difference  was  found,  for  the  exploratory  question,  Will  there  be  a significant  difference 
between  means  for  technology-related  occupations  and  non-technology  related  occupations  within  the  first 
administration  of  the  instrument? 

For  the  difference  between  means  of  technical  and  non-technical  occupations  in  the  first  administration  of 
the  OSES,  the  p value  was  .047.  As  the  sample  size  is  small,  a non-parametric  test,  the  Wilcoxon  Signed  Ranks  test 
was  also  run,  and  here  the  p value  was  .049,  leading  to  the  same  conclusion. 

This  difference  disappeared  in  the  second  administration  of  the  OSES,  where  scores  for  non-technology 
occupations  rose,  although  not  significantly.  Standard  deviations  for  technology-related  occupations,  particularly  in 
the  second  administration,  were  higher  than  for  non-technology-related  occupations: 

Table  1 Standard  Deviations 


Non- technology  1 Technology  1 


Non-technology  2 Technology  2 


1.6949 


1.8254 


1.6474 


2.0261 


These  results  are  somewhat  counter-intuitive  as  it  might  have  been  expected  that  technology  self  efficacy 
would  increase  after  participants  received  their  computers.  In  order  to  investigate  this  further,  responses  to  the  scale 
were  first  graphed  and  patterns  of  responses  were  analyzed,  in  order  to  generate  follow-up  questions.  Open-ended 
follow-up  qualitative  interviews  were  then  held. 

Three  types  of  typical  patterns  were  found:  one  subset  reached  the  ceiling  in  technology  self-efficacy  either 
in  the  second  administration,  or  in  both  administrations:  these  scores  either  rose,  or  stayed  the  same. 

Statements  from  this  group  in  follow-up  interviews  indicated  that  these  participants  wished  to  make  it  clear 
that  they  could  do  anything  if  they  had  the  opportunities  for  education  and  success  that  others  have.  These  members 
felt  that  they  were  capable  of  completing  the  education  for  occupations  requiring  high  levels  of  education,  but  felt 
others  in  society  were  unaware  of  their  ability. 

In  another  subset,  scores  fell,  sometimes  radically.  Interviews  with  this  group  indicated  that  after 
September  1 1 , some  participants  lost  interest  in  their  work  with  the  computer,  and  some  lost  faith  in  their  ability  to 
prepare  for  computer-related  jobs. 

With  a third  group,  response  levels  varied  based  on  the  specific  type  of  computer  career,  either  in  both 
administrations  or  in  the  second,  That  is,  a participant  might  score  high  on  a career  involving  computer  art,  but  low 
on  a career  involving  programming.  In  follow-up  interviews,  these  participants  indicated  that  their  greater 
understanding  of  technology  careers  had  led  them  to  give  more  precise  answers,  sometimes  resulting  in  an  overall 
lower  score. 


Looking  at  the  Participatory  Process 

With  regard  to  participant  goals,  localized  best  practices  were  found  regarding  technology  integration. 
Financial  goals  remained  problematic,  but  other  goals  defined  by  participants  as  pertaining  to  a better  life  were  met 
to  a degree.  As  one  participant  stated,  “I  will  stay  as  long  as  it  feeds  my  spirit.” 

Also  documented  in  the  focus  group  sessions  and  noted  in  the  Changes  to  the  Program  section  is  the 
refinement  of  the  participatory  action  research  process  over  the  course  of  the  study. 

Participatory  studies  may  more  commonly  limit  methods  to  ethnography:  in  this  study,  other  methods  were 
used  as  well,  but  introduced  with  care  and  always  with  participant  understanding  and  consent.  For  example,  the 
research  group  voiced  concerns  that  quantitative  methods  would  be  disempowering,  and  a discourse  analysis  has 
been  proposed  to  determine  if  the  self-efficacy  scale  is  less  empowering  with  this  population  than  an  open-ended 
questionnaire. 

As  a method,  discourse  analysis  was  not  seen  as  disempowering,  as  the  group  was  introduced  to  it  via 
Fairclough’s  call  for  lay  attention  to  discourse  as  a means  of  taking  power  (Fairclough,  1989.)  The  origins  of 
conversation  analysis  being  to  some  extent  in  ethnomethodology,  a field  in  which  ordinary  people  are  seen  as  able  to 
understand  their  own  discourse,  further  makes  this  method  approachable  to  discourse  analysis.  Finally,  the  place  of 
self  efficacy  in  psychology  as  introducing  the  importance  of  cognition  makes  this  concept  also  appropriate. 
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Discussion  and  Future  Research 

Generalizability  in  the  traditional  meaning  of  the  word  is  not  the  purpose  of  a study  that  takes  a qualitative 
perspective;  instead,  theory  is  generated  in  the  form  of  hypotheses  based  directly  on  data.  In  such  a study,  sufficient 
detail  is  ideally  presented  for  readers  to  be  able  to  make  the  decision  as  to  whether  or  not  the  findings  may  also 
generalize  to  particular  situations  with  which  the  researcher  is  familiar.  Such  detail  adds  to  the  credibility  of  the 
study.  In  this  brief  paper,  detail  is  necessarily  necessity  limited:  further  papers  are  planned,  and  in  the  meantime  it  is 
hoped  that  the  detailed  presentation  here  of  the  participatory  process  as  used  in  this  study  will  encourage  the  reader 
to  entertain  the  assumption  that  the  rest  of  the  study  may  likewise  be  credible. 

It  is  also  important  for  credibility  that  rapport  and  “buy-in”  on  the  part  of  participants  can  be  shown.  This 
study  sought  such  rapport  and  buy-in  by  following  participatory  action  research  principles.  As  all  participants  are 
co-researchers  with  an  equal  voice  in  decision  making,  rapport  was  a natural  outcome. 

Credibility  is  further  enhanced  by  triangulation,  or  the  use  of  multiple  methods  and/or  data  sources, 
especially  when  the  findings  of  the  methods  and  sources  converge,  as  they  do  here.  In  terms  of  the  purposes  of  the 
project  and  of  the  study,  a project  using  specific  means  of  technology  integration  (participatory  research,  peer 
mentoring,  focus  groups,  home-based  technology  integration)  was  examined  in  differing  ways  (discourse  analysis, 
field  notes,  and  a self-efficacy  scale)  to  yield  converging  results. 

The  results  point  to  the  usefulness  and  potential  for  success  of  peer  mentoring,  a process  that  is  eminently 
affordable  as  a means  of  instruction.  The  discourse  strategies  can  be  studied  further;  one  next  step  for  this  project 
will  be  to  incorporate  the  strategies  into  mentor  training. 

Within  the  participatory  process,  it  is  interesting  to  note  that  grassroots  participants,  after  five  years  of  work 
with  the  process,  were  able  to  transform  the  project  to  meet  the  challenges  of  scalability  and  indeed  to  further  refine 
the  participatory  process.  Further  study  of  the  nature  of  power  and  empowerment  for  such  a group  appears  to  be  in 
order. 

As  mentioned  earlier,  participants  themselves  have  suggested  a close  look  at  the  discourse  of  those  who  are 
taking  the  OSES,  in  order  to  determine  if  the  OSES  is  empowering  or  disempowering  as  an  educational  experience 

Another  direction  for  study,  and  currently  underway,  is  the  further  development  of  the  OSES  in  the 
direction  of  a Technology  Occupational  Self-Efficacy  Scale,  Important  here  will  be  the  identification  of  sets  of 
related  technology  occupations  that  are  of  varying  difficulty,  so  as  to  satisfy  Bandura's  requirement  that  items  on  the 
scale  present  varying  levels  of  challenges  (Bandura,  2001).  To  take  an  example  from  the  current  version  of  the  scale, 
"video  game  tester"  and  "video  game  designer"  require  varying  levels  of  training  or  education.  Since  technology- 
related  occupations  exist  across  various  subdomains-programming,  networking,  and  art,  for  example-it  will  be 
important  to  identify  the  various  technology-related  domains.  A related  avenue  of  future  research  will  be  the  further 
development  of  an  instructional  intervention  that  describes  the  technology  domains. 

Appendix  A 

Transcription  Conventions  Used  in  the  Floaters.org  Study 
Markup  is  based  on  the  work  of  Gail  Jefferson  (1974,  1984). 

Markup  Definition/Samples 

[ Left  bracket:  beginning  of  overlap  or  interruption 

S.  Yah  I was  in  one  of  those  technology  classes  [or 

A.  [Computer  class? 


Right  bracket  (optional):  end  of  overlap  or  interruption 


Equal  signs:  no  break  or  pause. 


==  Two  equal  signs  indicate  no  break  between  the  lines.  One  equal  sign  may  be  lined  up  above  the 

other;  or,  one  may  appear  at  the  end  of  one  line  and  one  at  the  beginning  of  the  next. 
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A.  one  day= 

S.  =yah  (.) 

T. 

Dash:  speech  is  cut  off. 

A.  All  in  the  box.  It  all  stays  in  the  box- 

no  ! Get  out!  Can  anybody  hear  me? 

yah  Underscoring  denotes  a louder  voice  or  other  stress,  as  in  a slight  rising  inflection. 

U.  Yea::h,  Pe:te! 

dit  Bolded  text  denotes  an  even  louder  voice  or  other  form  of  exaggeration: 

P.  dit  dit  dit  dit 

F.  That  said  it  closed  down  (.)  i:m  properly= 

: : Colons  indicate  prolongation  of  the  prior  sound.  The  longer  the  colon  row,  the  longer  the  prolongation. 

N.  She  lo::ves  using  the  computer. 

< Angle  brackets  (carets,  greater  than  signs)  surrounding  text  and  pointing  inward:  speech  is  said 

more  quickly  relative  to  other  utterances. 

S.  >very  cool  very  cool< 

<>  Angle  brackets  (carets,  greater  than  signs)  surrounding  text  and  pointing  outward:  speech  is  said 

more  slowly  relative  to  other  utterances. 

<and  you  ca:  :n't  kill  it> 

( ) Empty  parentheses  or  with  the  word  "inaudible"  indicate  something  not  heard  by  the  transcriber. 

(yah?)  Parenthesized  words  are  guesses  on  the  part  of  the  transcriber. 

Degree  signs  indicate  something  said  very  quietly. 

F.  °Let  me  show  you.° 

{ } Curly  brackets  indicate  context. 

{ Context:  mentor  is  looking  at  computer. } 

((  ))  Doubled  parentheses  contain  descriptions  of  sounds  other  than  speech. 
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Pauses  are  measured  approximately. 

New  lines  and  micropauses  (.)  are  about  .1  second. 

A number  within  parentheses,  such  as  (2.5),  is  also  approximate. 

Some  pauses  were  the  result  of  engagement  with  the  computer.  Where  this  engagement  was  audible,  it  was 
transcribed  with  an  explanation  in  double  parentheses,  as  for  example  ((clicking)). 

Silences  longer  than  a few  seconds  were  coded  by  approximate  length  or  by  an  explanation  within  double 
parentheses:  ((extended  silence)). 
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Secondary  School  Students’  Readiness  for  E-learning 


Murat  Ataizi 
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Eskisehir,  Turkey 

Computers  and  computer-based  technologies  have  made  an  incredible  impact  on  societies  and  cultures. 

This  impact  has  made  dramatic  changes  in  the  field  of  education  as  well.  Today,  computers  are  common  tools  in 
almost  every  school  and  are  being  used  increasingly  in  all  subject  areas.  Especially  universities  which  offer  distance 
education  programs  are  using  computers  and  online  technologies  as  a main  medium  to  deliver  instruction.  Also, 
primary  and  secondary  schools  offer  computer  related  courses  to  their  students  to  prepare  them  for  higher  education. 

The  purpose  of  this  study  is  to  reveal  the  extent  of  secondary  education  (Lycee)  students’  readiness  for  e- 
learning.  E-learning  as  a form  of  distance  education  has  become  one  of  the  essential  movements  in  higher  education. 
Almost  all  of  the  higher  education  institutions  in  Turkey  as  well  as  the  rest  of  the  world  have  been  trying  to  offer  e- 
learning  opportunities  for  their  students.  However  success  in  e-learning  largely  depends  on  learners’  e-readiness. 
Literature  provides  tools  and  variables  for  measuring  e-readiness.  For  instance,  level  of  students’  self-efficacy  for 
online  technologies  can  be  used  to  predict  their  success  in  e-learning. 

Background 

Distance  education  is  not  a new  approach  in  the  world  of  education.  For  many  years  various  types  of 
courses  and  programs  have  been  developed  and  delivered  for  the  distance  learners  in  the  world.  In  Turkey,  Anadolu 
University  has  been  doing  distance  education  for  more  than  twenty  years.  The  university,  now,  has  around 
1.000.000  distance  students  in  its  system.  The  student  population  is  so  big  and  will  be  much  bigger  in  a near  future 
because  of  the  rapid  growth  of  young  population  in  Turkey  who  can  not  get  higher  scores  from  the  university 
entrance  exam  and  or  want  to  be  distance  students  for  any  other  reasons. 

In  this  system,  the  university  has  been  using  mainly  a basic  text  based  system  to  deliver  information.  In 
addition  to  the  text  based  system,  the  university  has  been  using  television  and  radio  programs  (sometimes  live, 
basically  from  the  stored  video  and  sound)  to  support  the  information  basically  given  by  the  texts.  The  university  has 
begun  to  use  web  supported  information  delivery  systems  to  help  the  learners  for  better  understanding  the  contents. 
Also,  for  more  than  three  years  (started  in  2001);  the  university  has  some  fully  online  degree  programs  for  the 
distance  students. 

Use  of  the  Internet  for  learning  is  a new  issue  in  Turkish  Educational  System.  There  is  a big  need  to  know 
the  students  thoughts  and  attitudes  before  they  enter  to  the  system.  Especially  their  readiness  for 
e-learning  is  a big  issue  for  instructional  designers  to  develop  effective  and  efficient  courses  and  programs. 

Bandura  (1977)  was  the  first  user  of  the  term  self-efficacy.  He  has  defined  self-efficacy  as  “self-assessment 
of  ability  to  master  a task  or  achieve  mastery  over  a specific  situation  or  people’s  judgements  of  their  capabilities.  It 
is  concerns  not  with  the  skills  one  has  about  a subject  but  with  judgements  of  what  one  may  do  with  whatever  skills 
one  possesses”  (Bandura,  1986,  p.  391).  Therefore,  the  concept  of  self-efficacy  was  context  specific. 

The  term  self-efficacy  has  been  extended  to  particular  domains,  including  the  use  of  computers  or  computer 
self-efficacy.  According  to  Compeau  and  Higgins  (1995),  computer  self-efficacy  has  a major  impact  on  an 
individual’s  expectations  towards  using  computers.  In  addition  to  this,  Oliver  and  Shapiro  (1993)  said  that  an 
individual  who  did  not  see  himself  or  herself  as  a good  user  were  less  likely  to  use  computers.  Computer  self- 
efficacy  and  e-readiness  may  be  connected  in  this  context.  If  an  individual  feel  comfortable  himself  or  herself 
toward  the  computer,  it  could  be  said  that  the  individual  is  ready  to  use  computer  or  to  deal  with  computer  in  a 
specified  context. 

According  to  Arani  (2001),  several  research  studies  have  investigated  computer  self-efficacy,  using  a 
variety  of  measures  to  study  different  subject  groups.  The  computer  self-efficacy  scale  created  by  Murphy  et  al. 
(1989)  is  used  by  many  researchers.  The  Murphy  scale  was  developed  to  measure  individuals’  self  perceptions  of 
accomplishments  surrounding  particular  computer-related  skills  and  knowledge.  The  32-item  scale  is  composed  of 
three  different  levels  of  computing  skills:  Beginner’s  level,  advanced  level,  and  a level  associated  with  mainframe 
computers.  Miltiadou  and  Yu  (1999)  have  been  developed  online  technologies  self-efficacy  scale  and  tested  to 
validate  the  scale  on  330  college  level  students.  The  researchers  indicated  that  the  Cronbach’s  Coefficient  Alpha  for 
the  instrument  was  0.95.  With  minor  modifications,  items  from  Miltiadou  and  Yu  scale  was  chosen  and  used  to 
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gather  information  regarding  high  school  students’  readiness  for  e-leaming  in  this  study  to  inform  instructional 
designers  endeavouring  to  create  and  design  effective,  efficient,  appealing,  and  successful  Internet-delivered  courses 
for  new  generation  students. 

Wallace  (1999),  in  his  major  study  of  computer  self-efficacy,  has  argued  that  there  were  very  few  studies  at 
the  school  level,  and  he  suggested  that  it  would  be  useful  to  conduct  studies  of  computer-self  efficacy  in  specific 
subject  groups.  The  results  of  this  research  will  indicate  Turkish  high  school  students’  levels  of  e-readiness  and  the 
levels  of  self-efficacy  toward  the  computer  and  online  designed  courses  and  programs.  Peoples  who  are  designing 
the  university  level  courses  and  degree  programs  may  get  some  benefit  from  the  research. 

The  present  study  examined  self-efficacy  levels  of  secondary  education  (Lycee)  students’  toward  online 
technologies  to  be  able  to  measure  their  readiness  for  e-learning  in  higher  education. 

The  study  investigated  the  following  research  questions: 

1.  What  is  the  current  state  of  computer  skills  of  students? 

2.  Do  the  students’  characteristics  (gender,  grade  level,  school  type)  have  any  effect  on  their  computer 
self-efficacy? 


Method 

Participants 

Two  hundred  ninety  two  students  from  two  different  types  of  high  school  were  the  participants  of  the  study. 
The  majority  of  participants,  198  (67.8  percent),  were  from  Anadolu  high  school  that  has  selected  students  and  was 
teaching  intensively  science  related  courses  while  the  remaining  94  (32.2  percent)  students  were  from  ordinary  high 
school  that  has  normal  (50  percent  science,  50  percent  literature  and  social  courses)  curriculum. 

Procedures 

Data  was  collected  in  June,  2005.  Survey  was  distributed  and  collected  during  normally  scheduled  class 
times  and  took  10  to  15  minutes  to  complete. 

Instrument 

The  instrument  consisted  of  30,  5-point  Likert-scaled  items.  For  each  of  them,  students  were  asked  to 
indicate  their  level  of  confidence  from  l=not  confident  at  all  to  5=very  confident.  The  3.41  mean  score  identified  as 
the  expected  level  of  confidence  with  the  item  while  other  responses  enables  the  students  to  show  higher  or  lover 
level  of  confidence.  The  3.41  mean  average  was  determined  after  identifying  the  critical  level:  4 intervals/5 
categories  = 0.8.  The  instrument,  also,  has  four  subscales  1)  Internet  Competencies,  2)  Synchronous  Interaction,  3) 
Asynchronous  Interaction  I,  and  4)  Asynchronous  Interaction  II. 

Table  1:  Internet  Competencies  Subscale 

I feel  confident. . . 

1.  Opening  a web  browser  (e.g.  Explorer  or  Netscape) 

2.  Reading  text  from  a web  browser 

3.  Clicking  on  a link  to  reach  a specific  web  site 

4.  Accessing  a specific  web  site  by  typing  the  address  (URL) 

5.  Bookmarking  a web  site 

6.  Printing  a web  site 

7.  Conducting  an  Internet  search  using  one  or  more  keywords 

8.  Downloading  (saving)  an  image  from  a web  site  to  a disk 

9.  Copying  a block  of  text  from  a web  site  and  pasting  it  to  a 
document  in  a word  processor 

10.  Creating  a simple  web  page  with  text,  images,  and  links 
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Table  2:  Synchronous  Interaction  Subscale 

I feel  confident. . . 

11.  Providing  a nickname  within  a synchronous  chat  system  (if 
necessary ) 

12.  Reading  messages  from  one  or  more  members  of  the 
synchronous  chat  system 

13.  Answering  a message  or  providing  my  own  message  in  a 
synchronous  chat  system  (one-to-many  interaction ) 

14.  Interacting  privately  with  one  member  of  the  synchronous 
chat  system  ( one-to-one  interaction ) 


Table  3:  Asynchronous  Interaction  Subscale  I 

I feel  confident. . . 

15.  Logging  on  and  off  an  e-mail  system 

16.  Sending  an  e-mail  message  to  a specific  person  (one-to-one 
interaction) 

17.  Sending  one  e-mail  message  to  more  than  one  person  at  the 
same  time 

18.  Replying  to  an  e-mail  message 

19.  Forwarding  an  e-mail  message 

20.  Deleting  messages  received  via  e-mail 

21.  Creating  an  address  book 

22.  Saving  a file  attached  to  an  e-mail  message  to  a local  disk 
and  then  viewing  the  contents  of  that  file 

23.  Attaching  a fde  (image  or  text)  to  an  e-mail  message  and  then 
sending  it  off 


Table  4:  Asynchronous  Interaction  Subscale  II 

I feel  confident. . . 

24.  Signing  on  and  off  an  asynchronous  conferencing  system 

25.  Posting  a new  message  to  an  synchronous  conferencing 
system 

26.  Reading  a message  posted  on  an  asynchronous  conferencing 
system 

27.  Replying  to  a message  posted  on  asynchronous  conferencing 
system  so  that  all  members  can  view  it  (reply  to  all) 

28.  Replying  to  a message  posted  on  asynchronous  conferencing 
system  so  that  only  one  member  can  view  it  (reply  to  sender) 
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29.  Downloading  a file  from  an  asynchronous  conferencing 
system  to  a local  disk 

30.  Uploading  (sending)  a file  to  an  asynchronous  conferencing 
system 


Table  5:  Gender 


Gender 

Frequency 

Percent 

Male 

139 

47.6 

Female 

153 

52.4 

TOTAL 

292 

100 

Table  6:  School  type 

School  type 

Frequency 

Percent 

Ordinary  high  school 

94 

32.2 

Anadolu  high  school 

198 

67.8 

TOTAL 

55 

100 

Table  7:  Grade  level 

Grade 

Frequency 

Percent 

9th  grade 

180 

61.6 

10th  grade 

112 

38.4 

TOTAL 

55 

1 00 

The  mean  scores,  standard  deviations,  and  t- tests  analysis  were  used  to  interpret  the  data  gathered  via  the 
survey  instrument.  The  Cronbach’s  Coefficent  Alpha  for  the  instrument  was  0.95. 

Results  and  Discussions 

What  is  the  current  state  of  computer  skills  of  students? 

The  first  research  question  was  about  the  current  state  of  computer  skills  of  students.  Table  8 shows  overall 
mean  scores  of  the  students’  responses  to  each  subscale.  According  to  the  findings,  the  students  were  confident  with 
their  computer  skills  on  each  subscale.  First  skill  was  about  the  Internet  competencies  (M=3.86  > 

Meis  = 3.41).  From  this  finding,  it  can  be  said  that  the  students  were  feel  themselves  comfortable  with  the  Internet 
options  of  computer.  The  second  skill  was  about  synchronous  interaction  on  the  net  (M=3.80  > 

Meis  = 3.41).  From  this  finding,  it  can  be  said  that  the  students  were  comfortable  with  the  synchronous 
communication  via  the  computer.  The  third  skill  was  about  asynchronous  interaction  on  the  net  (M=3.92  > 
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Mels  = 3.41).  From  this  finding,  it  can  be  said  that  the  students  were  using  e-mail  regularly  and  comfortable  with  the 
asynchronous  communication.  The  last  skill  was  about  asynchronous  interaction  also  (M=3.46  > 

Meis  = 3.41).  From  this  finding,  it  can  be  said  that  the  students  were  using  computer  conferencing  system,  subscribe 
and  unsubscribe  from  groups  and  felt  comfortable  themselves  with  the  asynchronous  communication.  Overall,  the 
students  were  good  at  communicating  via  the  computer. 


Table  8:  Mean  and  standard  deviation  scores  of  confidence  subscales 


Subscales 

N 

M 

SD 

Internet  Competencies 

292 

3.86 

.95 

Synchronous  Interaction 

292 

3.80 

1.06 

Asynchronous  Interaction  I 

292 

3.92 

1.03 

Asynchronous  Interaction  II 

292 

3.46 

1.08 

Do  the  students 9 characteristics  ( gender , grade  level , school  type ) have  any  effect  on  their  computer  self-efficacy? 

The  second  question  of  the  study  investigated  the  differences  occur  in  the  students’  overall  computer  skills 
due  to  their  characteristics  such  as  gender,  grade  level,  and  school  type.  An  independent  sample  t-test  analysis  has 
been  conducted  to  see  if  gender  makes  any  differences  in  the  students’  computer  skills.  The  results  of  the  analysis 
summarised  in  table  9. 

According  to  the  results  male  students  scored  higher  than  female  students.  Internet  competencies  subscale 
was  significant  (t=  -2.18,  DF=290,  p=0.30)  and  asynchronous  interaction  II  was  significant  also  (t=  -2.18, 
DF=290,  p=0.30).  For  both  subscales  mean  number  gap  between  male  and  female  students  were  higher  than  other 
subscales.  Other  computer  skill  types  between  females  and  males  were  not  significant. 


Table  9:  t-test  results  for  gender  effect 


Subscales 

Gender 

N 

M 

SD 

Df 

Sig. 

(2-Tailed) 

Internet  Competencies 

Female 

153 

3.75 

.91 

290 

.030* 

Male 

139 

3.99 

.98 

.031* 

Synchronous  Interaction 

Female 

153 

3.72 

.98 

290 

.213 

Male 

139 

3.88 

1.14 

.216 

Asynchronous  Interaction  I 

Female 

153 

3.87 

.96 

290 

.394 

Male 

139 

3.97 

1.11 

.398 

Asynchronous  Interaction  II 

Female 

153 

3.33 

1.02 

290 

.030* 

Male 

139 

3.60 

1.12 

.031* 

An  independent  sample  t-test  analysis  has  been  conducted  to  see  if  school  types  make  any  differences  in  the 
students’  computer  skills.  The  results  of  the  analysis  summarised  in  table  10. 

According  to  the  results  asynchronous  interaction  II  was  significant  (t=  2.51,  DF=290,  p=0.13).  Other 
computer  skill  types  between  schools  were  not  significant. 

Table  10:  t-test  results  for  school  type  effect 
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Subscales 

School 

Type 

N 

M 

SD 

Df 

Sig. 

(2-Tailed) 

Internet  Competencies 

OHS 

94 

3.85 

.91 

290 

.850 

AHS 

198 

3.87 

.98 

.836 

Synchronous  Interaction 

OHS 

94 

3.87 

.98 

290 

.417 

AHS 

198 

3.76 

1.14 

.370 

Asynchronous  Interaction  I 

OHS 

94 

3.84 

.96 

290 

.388 

AHS 

198 

3.95 

1.11 

.337 

Asynchronous  Interaction  II 

OHS 

94 

3.69 

1.02 

290 

.013* 

AHS 

198 

3.35 

1.12 

.007* 

OHS:  Ordinary  high  school,  AHS:  Anadolu  high  school 


An  independent  sample  t-test  analysis  has  also  been  conducted  to  see  if  grade  level  makes  any  differences 
in  the  students’  computer  skills.  The  results  of  the  analysis  summarised  in  table  11. 

According  to  the  results  computer  skill  types  between  9th  and  10th  grades  were  not  significant.  But  the  10th 
graders  scored  higher  than  the  9th  graders.  The  reason  for  this  can  be  the  older  students  have  more  experienced  than 
the  younger  ones. 

Table  11:  t-test  results  for  grade  level  effect 


Subscales 

Grade  N M SD  Df  Sig. 

Level  (2-Tailed) 

Internet  Competencies 

9th  181  3.82  .90  290  .32 

grade 

10th  111  3.93  1.02  .33 

grade 

Synchronous  Interaction 

9th  181  3.78  .98  290  .79 

grade 

10th  111  3.82  1.18  .80 

grade 

Asynchronous  Interaction  I 

9th  181  3.84  .96  290  .10 

grade 

10,h  111  4.04  1.13  .11 

grade 

Asynchronous  Interaction  II 

9th  181  3.43  1.04  290  .49 

grade 

10th  111  3.51  1.14  .50 

grade 

Conclusions 

This  descriptive  study  reveals  self-efficacy  levels  of  secondary  education  (Lycee)  students’  toward  online 
technologies  to  be  able  to  measure  their  readiness  for  e-learning  in  higher  education.  This  research  is  just  a 
beginning  to  learn  and  understand  the  younger  generations’  skills  on  the  computer  technologies.  According  to 
findings,  high  school  students’  seems  to  be  ready  for  online  delivered  education.  As  expected,  high  school  students’ 
confidences  in  their  use  of  computers  are  quite  high  and  there  is  no  big  gap  between  male  and  female  students.  But 
the  pedagogical  issues  should  be  investigated  in  further  researches.  Online  courses  should  be  designed  to  deal  with  a 
wide  range  of  student  computer  skills  so  as  not  to  overwhelm  low  skill  students  with  higher  skill  levels. 
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Modular  Object-Oriented  Dynamic  Learning  Environment 
What  Open  Source  Has  To  Offer 


Pavlo  Antonenko 
Serkan  Toy 
Dale  Niederhauser 
Iowa  State  University 

Open  source  online  learning  environments  have  emerged  and  developed  over  the  past  10 
years.  In  this  paper  we  will  analyze  the  underlying  philosophy  and  features  of  MOODLE 
based  on  the  theoretical  framework  developed  by  Hannafin  and  Land  (2000). 
Psychological,  pedagogical,  technological,  cultural,  and  pragmatic  foundations  comprise 
the  framework  and  represent  the  major  points  of  our  analysis.  This  paper  is  intended  for 
instructional  designers,  distance  education  specialists,  K-12  and  college  instructors  who 
may  want  to  add  an  online  component  to  their  courses. 

As  we  enjoy  great  Advantages  from  the  Inventions  of  others  we 
should  be  glad  of  an  Opportunity  to  serve  others  by  any 
Invention  of  ours,  and  this  we  should  do  freely  and  generously. 

- Benjamin  Franklin 


Introduction 

Benjamin  Franklin’s  philosophy  resonates  in  the  recent  advent  of  open  source  software.  The  term  “open 
source”  refers  to  computer  programs  or  operating  systems  for  which  the  source  code  is  publicly  available.  (Johnson- 
Eilola,  2002)  The  definition  further  explains  that  inherent  in  the  open  source  philosophy  is  the  freedom  of  a 
distributed  community  of  programmers  to  modify  and  improve  the  code  (Perens,  1999). 

Open  source  software  promotes  the  use  of  technologically-neutral,  non-proprietary  tools  and  formats, 
which  allow  for  wide-spread  access.  According  to  the  Open  Source  Initiative  website  (2004),  the  major  reasons  for 
utilizing  open  source  software  include  free  distribution,  freedom  to  modify  the  software  to  meet  individual  needs, 
cross-platform  compatibility  and  universal  accessibility,  and  active  collaboration  to  improve  design.  These  factors 
bear  a special  significance  in  an  educational  setting.  As  Terry  Vessels  (2004)  puts  it: 

...Educators  have  been  called  upon  throughout  history  to  combat  censorship  imposed  by  various 
powers  over  the  flow  of  information.  The  censorship  being  applied  today  comes  in  the  form  of 
licenses  that  lock  away  the  tools  to  build  the  information  age  and  laws  that  limit  fair  use  in  ways 
that  are  unprecedented  in  the  modern  era  (f  2). 

The  open  source  movement  has  already  had  a significant  impact  in  the  business  world  (Wheeler,  2003),  and 
is  now  drawing  the  attention  of  educators  around  the  globe.  Distance  education  is  at  the  forefront  of  using  and 
creating  open  source  applications  in  education. 

Current  advances  in  open  source  online  learning  environments  are  a response  to  the  shortcomings  of 
commercial  products  like  WebCT  and  Blackboard.  One  such  weakness  is  a lack  of  flexibility  in  designing  and 
adding  customized  learning  modules.  With  commercial  products  one  can  only  include  elements  that  the  software 
designers  deemed  necessary  when  they  developed  the  program.  With  an  open  source  learning  environment  it  is 
possible  to  download  and  use  any  learning  module  one  might  find  on  any  open  source  software  website.  This  opens 
almost  limitless  capabilities  for  the  user  to  customize  the  application  by  choosing  from  a variety  of  options  for  e- 
mail,  discussion  boards,  chat,  online  quizzes,  and  all  the  other  elements  one  might  want  to  include. 

Further,  as  the  open  source  definition  suggests,  the  actual  code  can  be  modified  and  improved  to  meet 
individual  needs.  So,  if  the  user  decides  that  an  open  source  module  he  or  she  found  is  almost  perfect,  the  code  can 
be  modified  to  meet  his  or  her  needs.  Look,  feel  and  functionality  can  all  be  changed  since  the  code  can  be  easily 
accessed  and  modified. 
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As  to  pricing  of  commercial  products,  “companies  are  moving  toward  selling  campuswide  access  to 
software,  and  toward  setting  prices  based  on  the  number  of  students  each  college  enrolls.”  (Young,  2002)  According 
to  Young,  the  company's  [WebCT]  current  software  costs  $3,000  to  $30,000  annually,  depending  on  the  size  of 
the  institution  and  the  level  of  use  of  the  software.”  This  is  particularly  important  with  seemingly  continual 
decreases  in  federal  and  state  appropriations  for  higher  education.  All  open  source  software,  on  the  other  hand,  is 
available  free  of  charge  to  anyone  who  wants  to  use  it. 

Shortcomings  of  commercial  distance  education  software  have  prompted  the  development  of  a number  of 
open  source  online  learning  environments  such  as  MOODLE,  EduCreate,  Covidia,  and  LogiCampus.  Concurrently, 
the  open  source  concept  has  developed  to  the  point  that  even  the  tools  used  to  create  such  systems  are  open  source. 
For  example,  most  of  this  software  is  written  in  Hypertext  Preprocessor  (PHP)  an  open  source  alternative  to 
commercial  scripting  languages  and  make  use  of  open  source  relational  database  systems  like  MySQL.  They  can  be 
installed  on  almost  any  web  server — the  most  popular  being  Apache  (again,  open  source).  One  open  source  online 
learning  environment,  Modular  Object-Oriented  Dynamic  Learning  Environment  (MOODLE)  is  a highly  usable, 
reliable,  and  functional  alternative  to  popular  commercial  products  like  WebCT  and  Blackboard.  Developed  by 
Martin  Dougiamas,  a PhD  student  in  Computer  Science  and  Education  at  Curtin  University  of  Technology, 
Australia,  this  learning  environment  provides  the  considerable  flexibility  inherent  in  open  source  software  for 
designing  and  administering  Web-based  courses  to  meet  the  individual  needs  of  online  educators.  It  was  specifically 
developed  to  address  the  aforementioned  shortcomings  of  commercial  online  learning  environments — which  he  had 
used  and  supported  in  his  own  teaching  and  as  a technician  working  with  faculty.  In  his  own  words: 

...  It  started  in  the  90's  when  I was  a webmaster  at  Curtin  University  of  Technology  and  a system 
administrator  of  their  WebCT  installation.  I encountered  many  frustrations  with  the  WebCT  beast 
and  developed  an  itch  that  needed  scratching  - there  had  to  be  a better  way  (no,  not  Blackboard  :) 
(Dougiamas,  2004,  f 2). 


Analysis  of  MOODLE 

Dougiamas  decided  to  create  his  own  online  learning  environment  in  the  open  source  format  and  allow  the 
open  source  community  to  help  develop  and  refine  his  ideas.  MOODLE  was  designed  to  support  and  promote  users 
interested  in  developing  constructivist,  student-centered  learning  environments  (Dougiamas,  2004).  To  examine  this 
claim,  we  conducted  an  analysis  of  MOODLE  using  a framework  developed  by  Land  and  Hannafin  (2000)  which 
was  initially  designed  as  a guide  for  developing  constructivist  learning  environments.  According  to  the  authors, 
“Learning  environments,  directed  as  well  as  constructivist,  are  rooted  in  five  core  foundations:  psychological, 
pedagogical,  technological,  cultural,  and  pragmatic”  (Hannafin  & Land,  1997).  Figure  1 highlights  the  five 
components  of  these  core  foundations  as  applied  to  the  design  of  student-centered  learning  environments. 
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Figure  1:  Five  core  foundations  of  student-centered  learning  environments  (Hannafin  & Land,  1997) 


In  the  remainder  of  the  paper  we  will  relate  each  of  Hannafin  and  Land’s  core  foundations  to  the 
underlying  philosophy  and  principles  that  guided  the  development  of  MOODLE. 

Psychological  Foundations 

Psychological  foundations  address  research,  theory  and  practice  associated  with  how  people  think  and 
learn.  Our  examination  of  the  program  shell  and  standard  MOODLE  learning  modules  reflect  the  designers’  use  of 
several  important  considerations  from  cognitive  psychology: 

Situated  cognition  suggests  effective  learning  should  involve  immersing  students  in  authentic  activity  and 
culture  in  a real-world  learning  context.  (Brown,  Collins,  & Duguid,  1989).  Relevance  is  enhanced  through 
interconnected,  embedded  engagement  with  interesting  and  complex  tasks  situated  in  an  authentic  context.  Situated 
learning  integrates  four  critical  factors  that  maximize  student  learning  potential:  content,  context,  community,  and 
participation  (Stein,  1998).  MOODLE  learning  modules  allow  instructors  to  set  up  complex,  ill-defined  and 
authentic  tasks  in  real-life  contexts,  and  assign  roles  for  students  to  assume  in  the  solution  of  these  problems.  For 
example,  the  Workshop  module  allows  students  to  collaborate  on  the  design  of  possible  solutions  to  an  authentic 
problem  and  peer  assess  the  suggested  solutions.  The  instructor  might  decide  to  have  students  work  individually  or 
in  groups  to  determine  where  to  build  an  alternate  route  to  help  alleviate  traffic  accidents  at  a specific  intersection. 

In  the  Resource  module  the  instructor  can  scaffold  the  activity  by  providing  area  maps,  different  perspectives  on  the 
problem  (e.g.  interviews  with  Department  of  Transportation  experts),  and  sample  solutions  devised  by  other  states  or 
counties.  Students  can  brainstorm  possible  solutions  in  Chat  or  Discussion  forums,  then  present  their  alternate  routes 
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via  the  Workshop  module,  which  allows  for  peer  assessment.  Thus,  learners  will  compare  and  contrast  their 
solutions  and  select  the  best  one  based  on  the  group  discussions. 

Cognitive  flexibility  is  defined  as  “the  ability  to  adaptively  re-assemble  diverse  elements  of  knowledge  to 
fit  the  particular  needs  of  a given  understanding  or  problem-solving  situation”  (Spiro  & Jehng,  1990).  This  theory 
focuses  on  learning  in  complex  and  ill- structured  domains — which  represent  many  real-life  situations.  “A  central 
claim  of  cognitive  theory  is  that  revisiting  the  same  material,  at  different  times,  in  rearranged  contexts,  for  different 
purposes,  and  from  different  conceptual  perspectives  is  essential  for  attaining  the  goals  of  advanced  knowledge 
acquisition”  (Spiro,  Feltovich,  Jacobson,  & Coulson,  1991).  One  of  the  major  metaphors  employed  by  the  theory  is 
that  of  a “. . . criss-crossed  landscape,  with  its  suggestion  of  a non-linear  and  multi-dimensional  traversal  of  complex 
subject  matter,  returning  to  the  same  place  in  the  conceptual  landscape  on  different  occasions,  coming  from  different 
directions”  (Spiro,  Vispoel,  Schmitz,  Samarapungavan,  & Boerger  1987;  Wittgenstein,  1953).  MOODLE  supports 
the  “criss-crossing  of  the  conceptual  landscape”  through  the  Glossary  learning  module-a  unique  feature  that  allows 
users  (both  students  and  instructors)  to  create  an  online  hypertext-based  dictionary  that  is  created  on  the  fly  and 
updates  automatically  to  all  content  in  the  course  or  even  throughout  the  entire  portal.  This  module  allows  users  to 
consult  the  glossary  dynamically  while  navigating  through  the  lesson  content,  assignments,  or  even  discussion 
postings,  but,  perhaps  more  importantly,  students  take  ownership  for  their  learning  as  they  actively  construct  a 
richer,  more  complex  and  sophisticated  learning  environment. 

Pedagogical  Foundations 

Pedagogical  foundations  include  the  instructional  practices  that  the  designers  use.  They  are  grounded  in 
theories  of  learning  and  reflect  the  teaching  strategies  with  which  they  are  aligned.  MOODLE  developers  have 
explicitly  stated  that  the  design  of  the  software  is  grounded  in  constructivist  and  constructionist  instructional 
principles.  In  the  following  paragraph,  Dougiamas  (2004)  discusses  the  importance  of  pedagogy  and  encourages 
educators  to  adopt  the  constructivist  methodology: 

Once  you  are  thinking  about  all  these  [pedagogical]  issues,  it  helps  you  to  focus 
on  the  experiences  that  would  be  best  for  learning  from  the  learner's  point  of 
view,  rather  than  just  publishing  and  assessing  the  information  you  think  they 
need  to  know.  It  can  also  help  you  realise  how  each  participant  in  a course  can 
be  a teacher  as  well  as  a learner.  Your  job  as  a 'teacher'  can  change  from  being 
'the  source  of  knowledge'  to  being  an  influencer  and  role  model  of  class  culture, 
connecting  with  students  in  a personal  way  that  addresses  their  own  learning 
needs,  and  moderating  discussions  and  activities  in  a way  that  collectively  leads 
students  towards  the  learning  goals  of  the  class,  (f  13) 

Constructivism  implies  that  the  learner  links  new  information  with  existing  and  future- oriented  knowledge 
in  unique  and  meaningful  ways  (McCombs  & Whisler,  1997).  Social  constructivism,  a branch  of  this  theory, 
emphasizes  the  value  of  knowledge  that  is  built  socially  in  a learning  community.  Pioneered  by  theorists  like 
Vygotsky  (1978),  this  paradigm  argues  for  the  importance  of  culture  and  context  in  forming  understanding.  Learning 
is  not  a purely  internal  process,  nor  is  it  a passive  shaping  of  behaviors.  Vygotsky  favored  a concept  of  learning  as  a 
social  construct  which  is  mediated  by  language  via  social  discourse. 

MOODLE  promotes  social  discourse  in  learning  through  the  synchronous  and  asynchronous 
communication  modules  described  above.  Internal  support  for  introducing  groups  within  a class  of  students  in  the 
learning  environment  is  built  into  the  program.  Students  can  form  cohorts  themselves  or  the  instructor  can  moderate 
this  process.  Within  a cohort  students  work  cooperatively  and  engage  in  a more  individualized  interaction  with  one 
another.  Later,  cohorts  can  share  their  perspectives  in  a whole  class  discussion  and  continue  the  learning  process  as  a 
unified  group.  Although  the  program  supports  this  type  of  instruction,  the  art  of  combining  individual  activities  with 
cohort-based  or  whole-class  activities  is  one  of  the  factors  that  reflect  the  teaching  skills  of  the  instructor.  An 
instructor  may  choose  to  integrate  a discussion  forum,  chat  or  even  a private  two-way  dialogue  into  any  learning 
activity  of  the  course.  Wiki  is  the  module  that  gives  students  and  instructors  the  opportunity  to  collaborate  on  the 
design  of  hypertext  that  represents  the  knowledge  constructed  socially  by  the  learning  community  of  an  individual 
class.  A community  of  graduate  students,  for  example,  can  thus  work  together  on  a literature  review  for  a specific 
topic.  The  document  is  started  when  a student  makes  the  first  text  entry.  Other  students  modify  this  document  to 
develop  the  ideas  for  the  literature  review.  The  system  keeps  track  of  each  modifications  and  both  students  and 
instructor  have  the  opportunity  to  see  how  the  document  developed  over  time,  and  who  and  how  each  person 
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contributed.  Younger  learners  might  enter  and  revise  class  notes  in  a group  Wiki , compare  and  revise  the  notes  as  a 
class,  and  create  a useful  resource  for  each  other  and  for  future  students. 

Constructionism  (as  opposed  to  Constructivism)  asserts  that  that  knowledge  acquisition  is  particularly 
effective  when  constructing  something  for  others  to  experience.  Papert  (1991),  who  started  developing  this  concept 
in  the  1980s,  asserts  that  constructionism  “...shares  constructivism's  connotation  of  learning  as  "building  knowledge 
structures"  irrespective  of  the  circumstances  of  the  learning.  It  then  adds  the  idea  that  this  happens  especially 
felicitously  in  a context  where  the  learner  is  consciously  engaged  in  constructing  a public  entity,  whether  it's  a sand 
castle  on  the  beach  or  a theory  of  the  universe.”  Constructionism  stresses  the  importance  of  building  external 
artifacts  as  a means  to  more  effectively  construct  and  represent  the  inner  knowledge  structures.  The  importance  of 
learning  through  design  is  supported  by  the  research  on  children’s  development  of  strategies  and  collaboration  in 
video  game  design,  building  and  learning  with  programmable  bricks  (Kafai  & Resnick,  1996).  In  addition  to  the 
Glossary  and  Wiki  examples  provided  earlier,  MOODLE  supports  the  construction  of  artifacts  by  allowing  learners 
to  present  and  share  their  knowledge  in  a variety  of  different  formats,  including  multimedia  and  hypertext.  These 
products  can  then  be  shared  with  others  through  Discussion  Forum , Resource  or  Assignment  modules.  Further,  as 
described  above,  the  Workshop  module  promotes  social  construction  of  knowledge  artifacts  by  allowing  students  to 
collaborate  on  the  possible  solutions  to  ill- structured  real-world  problems  and  evaluate  them  in  the  peer  assessment 
activity. 

Technological  Foundations 

Technological  foundations  affect  how  media  can  support,  limit,  or  improve  the  pedagogy  of  the  learning 
environment.  According  to  Land  and  Hannafin  (2000),  “. . . technology  can  control  the  pacing  and  chunking  of 
information  where  cognitive  load  limitations  are  assumed. . .” 

MOODLE  supports  the  pedagogy  of  the  learning  environment  through  its  interactive,  collaborative  and 
reflective  modules.  For  example,  the  Journal  module  allows  the  instructor  to  ask  the  students  to  reflect  on  a 
particular  topic,  and  edit  and  refine  their  answer  over  time.  This  activity  promotes  self-assessment,  critical  thinking, 
and  metacogntition.  Learning  journals  entice  students  to  think  in  unconventional  ways  (Fulwiler  1987)  and  provide 
an  opportunity  to  both  develop  and  capture  reflection  in  the  learning  process  (Moon,  1999). 

Although  the  flexibility  of  hypertext  systems  is  powerful,  it  may  result  in  disorientation  problems  (Theng, 
1997)  and  information  overload  (Niederhauser,  Reynolds,  Salmen  & Skolmoski,  2000).  The  MOODLE  shell 
provides  a framework  for  presenting  modules  that  accounts  for  these  potential  problems.  It  allows  to  structure  and 
control  the  presentation  of  the  learning  material  and  decrease  the  risk  of  the  “lost  in  hyperspace”  problem  (Boyle  & 
Encamacion,  1994).  For  example,  each  page  of  the  portal  has  a quick-jump  drop-down  menu  that  allows  users  to 
navigate  the  system  more  efficiently.  Users  may  also  use  the  personalized  navigation  bar  at  the  top  of  each  page, 
which  tracks  and  shows  the  history  of  previously  viewed  pages  for  each  specific  user.  Extraneous  cognitive  load 
often  occurs  when  instructional  materials  require  learners  to  use  cognitive  resources  to  search  for  specific 
information  without  providing  any  scaffolds  or  quick  and  easy  access  to  relevant  resources  (van  Merrienboer, 
Kirschner,  Kester,  2003).  Information  overload  of  learners  is  reduced  through  built-in  support  for  adaptive  hypertext 
navigation  (Hook,  1997).  Navigation  in  MOODLE  is  further  enhanced  via  the  use  of  the  Latest  News  section,  which 
allows  instant  access  to  the  most  recent  discussion  postings,  news  entries  and  assignments.  A similar  function  is 
performed  by  the  Calendar  module  that  highlights  the  upcoming  events,  due  dates  and  other  information  through 
simple  mouse  rollovers. 

A major  strength  of  using  MOODLE  lies  in  the  inherent  nature  of  open  source  software  which  promotes 
customization.  With  only  a basic  knowledge  of  web  scripting  one  can  add  almost  any  open  source  stand-alone 
application  to  customize  an  online  learning  environment  to  meet  individual  needs.  For  example,  more  than  half  of 
the  modules,  visual  themes,  and  administration  features  available  in  the  current  version  of  the  program  have  been 
added  by  different  members  of  the  open  source  community.  This  brings  us  to  the  discussion  of  another  important 
foundation  of  learning  environments  - the  cultural  aspect  of  design. 

Cultural  Foundations 

Cultural  foundations  tend  to  reflect  the  prevailing  values  of  a learning  community.  For  instance,  one  might 
find  particular  values  such  as  back  to  basics,  interdisciplinary  learning,  or  global  society  in  a given  learning 
environment.  (Land  & Hannafin,  2000) 

The  central  ideas  in  the  MOODLE  culture  are  collaboration,  sharing  and  community.  They  are  represented 
in  Dougiamas’  (2004)  discussion  of  social  constructivism,  one  of  the  four  major  concepts  in  the  underlying 
philosophy  of  MOODLE: 
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[social  constructivism]  extends  the  above  ideas  into  a social  group  constructing  things  for  one 
another,  collaboratively  creating  a small  culture  of  shared  artifacts  with  shared  meanings.  When 
one  is  immersed  within  a culture  like  this,  one  is  learning  all  the  time  about  how  to  be  a part  of 
that  culture,  on  many  levels,  (f  9) 

Virtual  community  is  defined  as  a community  of  people  sharing  common  interests,  ideas,  and  feelings  over 
the  Internet  or  other  collaborative  networks  (Rheingold,  2000).  Members  of  a learning  community  also  share  the 
meanings  that  they  make  of  the  learning  material.  Social  exchanges  by  individual  students  are  an  important  part  of 
the  group  interaction  and  learning.  They  help  build  a sense  of  trust  and  respect  among  community  members  (Lally  & 
Barrett,  1999).  Students  in  the  MOODLE  learning  environment  form  a cultural  community  by  interacting  in 
synchronous  and  asynchronous  discussion  modules,  Journal  and  the  collaboration  tools  like  Workshop , Wiki  and 
Glossary  discussed  earlier. 

Another  aspect  of  community  building  associated  with  MOODLE  involves  those  who  are  working  to 
develop,  refine,  and  support  advancement  of  the  program.  The  collaborative  nature  of  designing  and  supporting 
open  source  applications  like  MOODLE  reflects  the  important  social  aspect  of  software  development  and 
knowledge  construction  by  integrating  the  diverse  perspectives  and  expertise  of  the  members  of  the  international 
community  working  together  to  improve  the  quality  of  the  software.  The  community  of  MOODLE  developers  and 
users  bring  multiple  perspectives  and  skills  and  share  their  views  on  online  learning  in  MOODLE  discussion  forums. 
This  international  collaborative  effort  results  in  a truly  socially  constructed  design  process,  which  enhances  the 
quality  of  the  software  from  both  the  pedagogical  and  technological  perspective.  For  example,  Williams  Castillo,  “a 
curious  developer”  from  Caracas,  Venezuela,  contributed  to  this  open  source  project  by  designing  the  Glossary 
module.  He  also  maintains  a discussion  forum  on  creative  uses  of  the  Glossary.  Further,  the  community  works 
collaboratively  to  provide  technical  support  for  all  of  its  members. 

Pragmatic  Foundations 

As  the  name  suggests,  these  foundations  are  concerned  with  doing  a reality  check.  How  does  the  learning 
environment  correspond  to  the  needs  of  target  audiences?  What  are  the  benefits,  and  what  are  the  limitations? 

MOODLE  is  an  open  source  online  learning  environment  that  is  developed  for  the  administrators  of  web- 
based  courses  for  K-12  and  university  instructors.  The  system  is  efficient  and  features  cross-platform  compatibility 
and  a low-tech  browser  interface.  A highly  relevant  aspect  of  MOODLE  for  educators  is  that  it  is  available  as  a free 
download  on  the  Internet  and  can  be  installed  in  an  hour.  Even  though  one  can  administer  the  installation  of 
MOODLE  with  just  a basic  knowledge  of  web  interfaces,  one  should  probably  have  experience  managing  database- 
driven  dynamic  websites. 

With  the  software  installed  the  creation  and  management  of  the  learning  environment  can  be  performed  by 
a person  with  limited  technological  expertise.  MOODLE’ s creators  realize  that  educators  are  not  high-end 
developers  and  therefore  all  the  administration  is  performed  through  a simple,  intuitive  graphical  user  interface 
(Figure  2).  Help  buttons  are  included  for  every  component  of  the  administration  menu  and  provide  guidance  for 
novice  users  of  the  system. 

Most  text  entry  areas  in  MOODLE  such  as  resources,  forum  postings,  assignments  etc.  can  be  edited  using 
an  embedded  WYSIWYG  (What  You  See  Is  What  You  Get)  HTML  editor.  The  administrator  (instructor)  can  allow 
or  prohibit  students  to  modify  specific  parts  of  the  course,  like  journal  entries.  Further,  MOODLE  courses  can  be 
categorized  and  searched,  allowing  one  MOODLE  installation  to  support  thousands  of  courses  and  function  as  a 
campus  edition.  Plug-in  activity  modules  can  be  added  to  existing  MOODLE  installations  and  enhance  the  existing 
structure  of  the  courses.  Customizable  themes  allow  the  administrator  to  customize  the  site  colors,  fonts,  layout,  and 
other  features  to  suit  individual  needs.  MOODLE  language  packs  allow  full  localization  to  43  languages.  Even  the 
language  packs  can  be  easily  edited  using  a built-in  web-based  editor.  [Insert  Figure  2 about  here] 

Technical  support  is  freely  available  on  the  Web  and  is  provided  by  MOODLE  developers  and  users 
through  discussion  forums  and  Frequently  Asked  Questions  section.  Each  learning  module  is  supported  by 
a separate  discussion  forum  containing  tips  and  tricks,  teaching  strategies,  learning  standards,  course 
formats,  and  advice  on  how  to  build  a strong  learning  community. 

Conclusion 

Open  source  software  has  become  increasingly  popular  in  many  areas.  One  such  application,  MOODLE, 
provides  a constructivist  learning  environment  that  makes  a significant  contribution  to  enhance  web-based  learning. 

As  this  paper  demonstrates,  the  design  of  MOODLE  integrates  general  principles  of  constructivist  learning 
and  provides  an  online  learning  context  that  supports  student-centered  pedagogy.  This  system  is  grounded  in 
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situated  cognition  and  cognitive  flexibility  theory  that  provide  the  opportunity  for  an  instructor  to  create  a 
constructivist  and  constructionist  environment  to  enhance  teaching  and  learning.  The  capabilities  to  design  student- 
centered  learning  are  embedded  in  MOODLE  and  its  modules;  however,  it  is  ultimately  the  responsibility  of  the 
instructor  to  make  good  use  of  them. 

The  MOODLE  project  indicates  the  growing  interest  of  educators  and  open  source  programmers  in  joining 
their  efforts  to  improve  the  quality  and  reduce  the  cost  of  education.  Since  it  is  distributed  under  the  General  Public 
License,  MOODLE  can  be  easily  modified  to  meet  individual  needs.  Lurther,  development,  customization,  and 
support  are  all  completed  as  part  of  the  community  effort  to  improve  online  teaching.  This  open  source  application 
provides  an  effective  and  cost-efficient  alternative  to  expensive  commercial  software  packages  for  those  interested 
in  joining  the  movement  to  provide  high  quality  constructivist-based  educational  experiences  in  the  online 
environment. 


Figure  2:  Screenshot  of  MOODLE  course  administration  page 
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MIDI  Magic:  Exploring  the  New  World  of  Digital  Music 

Tom  Atkinson 

Central  Missouri  State  University 

As  the  song  goes,  “Don’t  know  much  about  His-to-ry,  don’t  know  much  about  Tech-no-lo-gy!”  But  what 
we  do  know  is  that  music  technology  now  plays  a cmcial  role  in  most  schools.  Fully  integrating  this  technology 
requires  much  greater  awareness.  From  Mozart  to  Madonna,  technology  has  forever  changed  the  field  of  music. 
Although  accessing  music  through  the  Web  and  through  digital  storage  devices  has  been  remarkably  significant, 
perhaps  to  an  even  greater  extent,  music  synthesizers  and  editing  software  have  dramatically  changed  the  very 
nature  of  music.  Technology  provides  powerful  aids  to  composing,  notating,  editing,  and  performing  music  that 
even  elementary  school  students  can  learn  to  use. 

If  you've  ever  had  an  original  song  idea  in  your  head  and  wished  you  could  have  it  performed,  MIDI  is  the 
way  to  do  it.  All  you  need  is  a MIDI  Sequencer,  plus  a MIDI  instrument  to  enter  notes.  You  can  also  use  MIDI 
Notation  software  to  place  notes  on  a musical  staff  without  playing  them  at  all.  You  can  start  with  just  a melody  and 
then  add  backing  chords,  bass,  and  rhythm  later,  or  add  instruments  in  any  order  you  like.  If  you  make  mistake,  you 
can  change  it  without  having  to  play  the  part  all  over  again.  You  can  also  make  entire  sections  repeat  without 
playing  them  again.  And  you  can  rearrange  and  re-orchestrate  your  song  as  many  times  as  you  like. 

Many  people  enjoy  arranging  and  orchestrating  music  as  much  as  performing  it.  There  are  MIDI  files 
available  for  songs  from  every  style  of  music,  as  well  as,  software  programs  that  generate  the  basic  rhythm  and 
chord  patterns  that  define  specific  styles  that  you  can  use  to  create  your  own  arrangements  and  orchestrations.  Just 
change  the  instrumentation,  add  a verse  or  chorus  here  or  there,  even  put  in  your  own  original  phrase  or  section.  All 
of  this  is  easy  to  do  with  MIDI.  You  can  also  share  your  arrangements  with  others,  who  can  then  rearrange  them. 
Many  people  download  MIDI  files  from  the  Internet  and  arrange  them  to  fit  their  own  needs. 

Band-in-a-Box  (Image  1),  described  as  an  intelligent  accompaniment  software,  is  a powerful  and  creative 
music  composition  tool  for  exploring  and  developing  musical  ideas  with  near-instantaneous  feedback.  It  contains 
features  to  display  notation,  enter  lyrics,  create  melodies,  add  harmonization,  and  program  a variety  of  musical 
styles.  The  Soloist  generates  professional  quality  solos  over  any  chord  progression.  The  Melodist  creates  songs  from 
scratch  with  chords,  melodies,  intros,  solos,  and  even  a title. 

Band-in-a-Box  can  add  recordings  of  acoustic  instruments  or  voices  to  the  composition  with  special  effects 
processing.  Its  built-in  audio  harmonies  can  turn  an  audio  track  into  multiple  harmony  parts  and  even  adjust  its  pitch 
by  tracking  the  Band-in-a-Box  melody  track.  Digital  audio  features  make  Band-in-a-Box  the  perfect  tool  for 
creating,  playing,  and  recording  music  with  MIDI,  vocals,  and  acoustic  instruments.  You  can  print  out  your  musical 
arrangement  with  repeats  and  endings,  DC  markings  and  codas,  or  save  it  as  a graphic  file  for  Web  publication  or 
even  e-mail  it  to  a friend.  And  when  you're  ready  to  let  others  hear  your  composition,  you  can  burn  it  directly  to  an 
audio  CD.  Save  your  composition  as  a Windows  Media  File  or  other  compressed  formats  for  a file  that's  "Internet 
ready."  It’s  a great  way  to  create  your  own  backup  band! 

PowerTracks  Pro  Audio  (Image  2)  provides  a fully-featured  MIDI  and  digital  audio  sequencer  and  multi- 
track recorder.  Unlike  Band-in-a-Box,  you  do  not  create  music  in  PowerTracks  by  typing  in  the  chords  to  a song. 
Rather,  you  'layer'  tracks  of  MIDI  and  digital  audio,  each  of  which  must  be  recorded  from  scratch.  This  takes  longer, 
but  in  exchange  you  have  much  more  control  over  the  nuances  of  your  music.  Band-in-a-Box  does  not  allow  you  to 
edit  the  individual  events  of  the  accompaniment  tracks.  This  is  because  the  program  generates  the  accompaniment 
for  you,  and  it  is  different  every  time  you  play  your  song.  PowerTracks  also  comes  loaded  with  a host  of  effects  to 
help  you  put  the  subtle,  finishing  touches  on  your  work.  Use  a bit  of  reverb  to  create  a 'spacious'  feel;  add  some 
chorus  and  distortion  to  enhance  the  'grind'  of  your  guitar  tracks;  fiddle  with  the  compressor  to  give  your  drums  that 
extra  'punch.'  PowerTracks  (and  patience)  are  all  you  need  to  infuse  your  songs  with  a refined,  'studio'  feel. 

After  creating  music,  playback  usually  occurs  through  devices  called  synthesizers.  Although  most 
computers  have  built-in  synthesizer  capability  on  their  sound  cards,  more  serious  performers  use  specialized 
hardware  or  software  to  generate  the  sounds  through  their  computer.  Musical-Instrument-Digital-Interface  (MIDI) 
is  a technology  that  represents  music  in  digital  form.  Unlike  other  digital  music  technologies  such  as  MP3  and  CDs, 
MIDI  messages  contain  individual  instructions  for  playing  each  individual  note  of  each  individual  instrument.  So 
with  MIDI  it  is  actually  possible  to  change  just  one  note  in  a song,  or  to  orchestrate  and  entire  song  with  entirely 
different  instruments.  Since  each  musical  part  in  a MIDI  performance  is  separate  from  the  rest,  it’s  easy  to  listen  to  a 
single  instrument  and  study  it  for  educational  purposes,  or  to  mute  individual  instruments  in  a song  so  that  you  can 
play  that  part  yourself.  Hardware  synthesizers  can  be  expensive  but  software  alternatives  are  quite  inexpensive  or 
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even  included  with  most  arranging  and  composition  programs.  For  example,  the  Edirol  Virtual  Sound  Canvas 
(Image  3)  allows  you  to  customize  playback  of  MIDI  files  using  the  same  sounds  and  for  a fraction  of  the  cost  of 
hardware  synthesizers.  You  can  even  export  your  finished  files  into  popular  multi-media  application  formats. 

From  beginner  to  the  professional,  technology  has  forever  changed  the  way  we  create  music. 
Experimentation  with  music  scanning  and  pattern  recognition  software  may  soon  make  it  possible  to  capture  not 
only  existing  scores  and  recordings  but  allow  even  non-musicians  to  explore  their  ideas  and  produce  unique 
arrangements  and  compositions,  all  with  simple  clicks  of  a mouse! 

Product  References 

BIAB  http://www.pgmusic.com/ 

PowerTracks  http://www.pgmusic.com/powertracks.htm 

Edirol  Synth  http://www.edirol.com/products/software.html 
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Background 

The  use  of  animation  and  audio  as  a virtual  panacea  for  everything  from  advertising  to  educational  videos 
and  instruction  has  created  the  presumption  that  any  materials  that  use  them  ‘must  be  better!’  Now  doubt  that  the 
addition  of  animation  can  improve  message  delivery  on  a number  of  scales,  but  the  use  of  animation  regardless  of 
message  and  with  little  concern  for  systematic  placement  may  be  causing  more  harm  than  good.  Combined  with  the 
increasing  prevalence  of  computing  technology  and  increasing  ease  of  development  on  standard  workstations  the 
integration  of  animation  in  web-based  instruction  is  a more  realistic  possibility.  This  study  explores  the  effect  of 
animation  on  higher  order  educational  objective  achievement  in  a web-based,  self-paced  programmed  instructional 
unit  on  the  human  heart  and  its  functions  for  undergraduate  students  with  majors  outside  of  the  life  sciences. 

Introduction 

In  instruction  the  use  of  pictorial  media  has  long  been  considered  to  be  an  important  instructional  variable 
supported  by  a number  of  theoretical  considerations.  Kulik  and  Kulik  (1985)  reported  that  computer-based 
instruction  enhances  learning  and  fosters  positive  attitudes  toward  instruction  with  college  students.  Unfortunately 
most  of  the  studies  that  have  been  carried  out  examining  animation  in  computer-based  and  web-based  instruction 
have  been  cursed  with  confounding  and  poor  designs.  (Rieber,  1991;  Dwyer  1978;  Park  & Hopkins  1993)  There  are 
various  reasons  for  the  conflicting  results  and  inconclusive  results.  Dwyer  (1978)  pointed  out  that  one  of  the  major 
problems  with  media  research  was  that  it  didn’t  describe  the  types  of  learning  tasks  or  objectives  that  it  expected  the 
participants  to  achieve.  Sadly  upon  review  of  available  literature  dealing  with  animation  in  computer-based  and 
web-based  instruction  this  still  appears  to  be  the  case.  In  the  subsequent  literature  review  this  author  makes  a case 
that  not  much  has  changed  in  the  past  25  plus  years  in  media  research  since  Dwyer’ s observation.  Further  the 
questions  of  effectiveness  in  practice  and  convergence  of  theory  remain  largely  inconclusive. 

Definition 

With  a lack  of  definition  of  terms  being  one  of  the  major  criticisms  of  the  research  reviewed  in  the 
subsequent  pages  the  researchers  will  explicate  their  operating  definitions  for  animation.  Animation  is  a sequence 
of  images  played  in  rapid  succession  such  that  to  the  human  eye  the  result  is  apparent  motion  (Park  & Gittelman, 
1992)  and  is  generally  used  in  instructional  materials  for  one  of  three  purposes:  attention-gaining  / attention- 
direction,  presentation,  and  practice  (Rieber,  1990). 

Literature  Review 

Paivio’s  dual  coding  theory  (DCT)  asserts  that  images  and  verbal  processes  together  determine  learning 
and  memory  performance  (Clark  & Paivio,  1991).  According  to  Paivio’s  (1971,  1986)  and  later  Clark  and  Paivio’s 
(1991)  explanation,  using  the  information  processing  model  and  the  spread  of  activation  in  the  brain,  the  links 
between  verbal  and  non-verbal  symbolic  storage  can  trigger  each  other, 44 . . .this  spreading  activation  results  in 
complex  patterns  of  arousal  among  the  representations  in  the  network.”  (Clark  & Paivio,  1991,  p.  154)  Further, 
Clark  and  Paivio  propose  that  both  types  of  mental  representations  have  dedicated  channels  for  the  processing  and 
encoding  of  information.  Tulving  (1976)  suggests  that  information  can  be  processed  on  several  levels  in  parallel 
processes  and  a logical  extension  of  this  using  information  processing  as  a guide  is  that  the  parallel  processing  can 
aid  in  the  transfer  of  information  into  long-term  storage.  The  concept  of  parallel  processing  is  not  new  to  cognitive 
psychology,  and  will  be  generally  accepted  by  the  researchers.  What  is  of  interest  to  the  researchers  stems  from  the 
apparent  inconsistencies  in  the  literature  regarding  the  use  of  animated  sequences  to  facilitate  learning  or 
achievement. 
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Animation  Studies  and  Findings 

Park  and  Hopkins  (1993)  suggest  that  educational  research  on  visual  displays  evolved  from  two  distinct 
camps:  the  behaviorist  with  work  by  Guthrie,  Skinner,  and  Thorndike  on  eliciting  desired  responses  after  some 
stimulus,  and  the  cognitive  with  work  by  Paivio  and  his  Dual  Coding  Theory  (DCT).  Tulving  and  his  concept  of  3 
major  types  of  memory:  episodic,  semantic,  and  procedural  would  also  fall  under  the  cognitive  branch  in  this 
division  as  well.  (Gredler,  2001,  p.  171) 

In  Rieber’s  1990  review  of  12  studies  spanning  25  years,  he  found  inconsistent  results  in  the  effects  of 
animation  on  achievement  and  learning  by  extension.  Similarly,  Park  and  Hopkins  (1993)  reviewed  25  empirical 
studies,  17  of  which  dealing  directly  with  computer  based  instruction  and  also  reported  mixed  results.  Dwyer  and 
Dwyer  in  a 2003  presentation  reviewed  5 animation  based  studies  using  similar  content  and  identical  assessment 
tools  on  a total  of  781  subjects,  and  found  only  three  cases  out  of  the  72  examined  where  animation  showed 
significant  benefits  over  static  visuals.  Owens  and  Dwyer  (2003)  in  an  unpublished  study  actually  found  animation 
to  be  less  effective  than  static  visuals  at  higher  levels  of  learning. 

In  a study  published  in  1988  by  Reiber  and  Hannafin  looking  at  the  effects  of  animated  or  textual  orienting 
activities  on  learning  in  computer  based  instruction  with  fourth,  fifth,  and  sixth  graders,  they  report  that  neither  text- 
based  or  animated  activities  were  powerful  influences  on  learning.  This  studies  content  was  based  on  Newton’s  laws 
of  motion,  and  the  authors  report  that  a 24  item  posttest  was  administered  with  a KR-20  reliability  of  .83  overall. 
While  not  explained  they  also  report  that  validity  was  established  through  expert  review  by  independent  science 
teachers. 

In  a study  published  one  year  later  in  1989,  Reiber  ran  another  factorial  study  3x2x2  looking  at  more 
factors  comparing  graphic  type  (animation,  static  graphic,  not  graphic)  and  text  type  (text,  no-text)  and  practice  type 
(relevant,  irrelevant).  Additionally  within  subject  he  examined  factual  verses  application  objectives  and  near  verses 
far  transfer.  The  overall  reliability  of  the  improved  dependent  measures  was  .91.  In  this  study  “The  lack  of  main 
effects  among  the  embedded  elaboration  conditions  was  surprising.”  (Reiber,  1989,  p.  439)  Two  years  later  Reiber 
(1991)  ran  a simple  version  of  the  study  looking  only  at  graphic  type  (static  graphics  verses  animated  graphics)  and 
practice  type  (simulation  verses  simulation  with  questions)  found  significant  effects  in  favor  of  animation  for  near 
transfer  on  incidental  and  intentional  questions. 

In  a 1998  animation  study  conducted  by  Park  using  a computer  based  instructional  unit  on  electronic  circuit 
repair  he  reports  that  static  graphics  with  motion  cues  can  be  used  instead  of  full  graphical  animation  as  they  both 
were  equally  effective  on  the  performance  and  transfer  tests  that  were  administered.  Will  this  finding  may  well  be 
true,  Park  did  not  identify  the  level  of  educational  objectives  measured  or  report  any  reliability  or  efforts  to  support 
validity  on  his  dependent  measures.  If  the  dependent  measures  were  measuring  factual  or  conceptually  based 
knowledge  than  the  finding  may  offer  some  direction  an  support  but  with  out  it  being  reported  the  readers  are  left  to 
their  own  devices  and  field  suffers  once  again. 

“Although  much  research  has  been  done  on  the  effectiveness  of  static  visuals  (Dwyer,  1978),  little  research 
has  conducted  on  animation’s  instructional  effects.  Empirical  data  that  are  available  are  inconsistent.”  (Reiber, 

1990,  p.  78)  In  the  almost  15  years  since  Reiber  wrote  this  statement,  little  has  changed.  More  studies  have  been 
run,  but  a general  consensus  has  yet  to  be  reached.  This  lack  of  consistency  stems  from  several  different  sources  in 
both  internal  and  external  validity  issues.  There  are  the  more  obvious  issues  of  poor  study  design,  varying  and 
insufficient  sample  sizes,  issues  with  content  relevance,  lack  of  systematic  process  for  placement  of  treatments,  use 
of  assessments  with  out  evidence  of  -or  even  reported  assessment  reliability  and  content  validity.  Even  with  all  of 
these  issues  surrounding  the  existing  literature,  one  of  the  most  egregious  errors  revolves  around  the  lack  of 
definition  of  the  types  or  methods  of  animation  used  and  the  levels  of  instructional  objectives  that  were  being 
addressed  in  the  studies.  The  failure  of  previous  researchers  to  provide  reasoning  for  placement  of  the  interventions 
casts  yet  more  doubt  on  use  and  application  of  reported  results.  With  out  a systematic  process  for  placement  of  the 
animation  the  net  effect  may  well  be  supporting  an  instructional  objective  that  is  already  sufficiently  addressed  in 
the  instruction  thereby  wasting  the  time  and  attentional  resources  of  the  participants  as  they  would  have  responded 
with  the  correct  answer  before  the  added  stimulus  was  introduced. 

Literature  Review  Summary 

The  numerous  studies  addressing  animation  and  its  effects  are  asking  good  questions,  but  are  for  the  most 
part  lacking  in  execution  in  some  major  areas  of  concern.  Gagne  (1985)  in  his  book  entitled  The  Conditions  of 
Learning,  proposed  that  there  are  different  types  of  learning  and  that  each  type  of  learning  requires  a different 
approach.  The  problem  remains  that  while  all  the  previously  mentioned  studies  are  asking  good  questions  they  are 
not  providing  sufficiently  based  answers.  The  need  for  the  systematic  placement  of  independent  variables,  which 
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previous  studies  lack,  and  the  use  of  sound  instruments  with  reported  reliability,  which  previous  studies  also  lack,  is 
paramount  in  accurate  interpretation  of  results. 

How  This  Study  is  Different 

Where  this  study  set  itself  apart  from  the  previous  studies  in  the  literature  is  through  the  systematic 
placement  of  the  animation  in  a programmed  instructional  unit.  Further,  in  this  study  the  researchers  have  gone  to 
great  pains  in  the  pilot  studies  to  refine  the  programmed  instruction  so  that  the  participants  have  the  necessary 
factual  and  conceptual  knowledge  to  build  upon  for  higher  order  learning  to  take  place.  “An  awareness  of  the  fact 
that  that  there  are  different  kinds  of  educational  objectives  each  requiring  specific  prerequisites  is  crucial  to 
educators  who  aspire  to  employ  the  visual  media  effectively”  (Dwyer,  1978,  p.  43).  Additionally  this  study  used 
instruments  with  previously  reported  reliability  and  demonstrated  discriminatory  power  for  its  dependent  measures. 

It  is  hypothesized  that  the  use  of  the  animation  can  reduce  the  overall  cognitive  load  on  the  participant  there  by 
allowing  them  to  queue  to  the  import  information  in  the  instruction  and  process  it  more  effectively. 

Statement  of  Purpose 

Specifically,  this  study  sought  to  examine  the  effectiveness  with  which  different  types  of  stimuli,  varied 
animation  strategies,  can  be  used  to  complement  a web-based  programmed  instruction  unit  to  improve  learner 
achievement  on  four  different  types  of  educational  objectives. 

Design  and  Methodology 

Eight-Eight  undergraduate  students  enrolled  in  lower-division  management,  educational  psychology,  and 
information  sciences  technology  classes  were  randomly  assigned  to  one  of  three  treatments  in  a randomized  1X3 
post-test  only  experimental  design.  The  type  of  animation  strategy  used  was  considered  to  be  the  independent 
variable  with  three  levels  (control  with  base  animation  employed,  simple  level  with  base  animation  and  simple 
reveal,  and  complex  level  with  base  animation  and  progressive  reveal).  Dependent  variables  were  the  scores 
achieved  on  the  criterion  measures  by  the  participants.  Participation  in  the  study  was  voluntary  and  at  the 
recruitment  sessions  the  students  were  able  to  select  their  preference  of  times  to  report  to  a list  of  labs  were 
researchers  would  be  waiting. 

Systematic  Placement  and  Development 

Two  pilot  studies  and  the  current  study  were  run  from  September  2003  through  September  2004  to 
systematically  apply  the  effective  use  of  the  animated  stimulus  in  the  final  iteration  reported  here.  A brief 
explanation  of  the  two  pilot  studies  and  treatments  development  will  be  covered  in  the  following  section. 

Pilot  Study  #1:  (n  = 12)  This  first  pilot  and  genesis  for  the  larger 
study  was  done  in  response  to  criticism  in  the  literature  that  one  of 
the  reasons  for  conflicting  results  was  that  it  was  unreasonable  to 
expect  any  achievement  differences  at  higher  levels  of  educational 
objectives  if  the  lower  levels  still  were  not  addressed  in  sufficient 
detail  as  to  achieve  an  acceptable  score  (average  of  90%). The  first 
pilot  study  was  conducted  to  facilitate  the  development  of  the  future 
instructional  treatments  and  test  the  programmed  instructional  units 
effectiveness.  This  pilot  study  took  the  instructional  booklet 
developed  by  Dwyer  and  Lamberski  in  1977  and  ported  the 
treatment  over  to  a web-based  instructional  unit  and  added  five 
quizzes  to  make  it  into  a programmed  instructional  unit.  There  was 
no  use  of  animation  and  only  a recreation  of  the  static  visuals  used 
in  the  original  paper  based  instruction  (figure  1).  The  purpose  was 
to  identify  items  with  a high  of  difficulty  (.60  or  below  on  the  drawing  and  identification  tests)  to  determine  areas 
where  the  static  visual  instruction  might  be  complemented  by  a refined  programmed  instructional  unit  to  further 
improve  student  achievement. 

Pilot  Study  #2:  (n  = 138)  This  second  pilot  built  off  of  the  lessons  learning  in  first  pilot  on  the  programmed 
instructional  unit.  Adaptations  and  corrections  were  made  in  the  programmed  instruction  in  hopes  of  pushing 
achievement  even  higher.  Additionally  there  were  questions  to  the  effectiveness  of  our  programmed  instructional 
unit.  If  you  are  building  a structure  you  need  a good  foundation,  the  factual  and  conceptual  information  is  that  base 
for  higher  order  objectives  was  specifically  stressed  in  the  programmed  instruction.  Simple  (base)  animations  were 
developed  using  flash  and  fireworks  for  the  graphical  development.  In  this  study  three  levels  were  established:  First 


Figure  1.  Sample  Pilot  Screen  Shot 
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there  was  the  instructional  script  from  Dwyer  and  Lamberski  (1977),  with  static  visuals  only  totaling  20  content 
pages.  The  second  treatment  (figure  2)  was  the  programmed  instructional  version  with  the  same  script  broken  down 
into  no  more  than  2 parts  or  concepts  per  webpage  for  a total  of  28  content  pages  with  embedded  quizzes  after  every 
5 or  six  parts  of  the  heart  were  covered  in  the  first  17  frames.  The  third  treatment  (figure  3)  has  the  same  breakdown 
and  structure  as  the  programmed  instructional  unit,  but  added  basic  animation  to  help  facilitate  and  reduce  the 
information  processing  load  on  the  part  of  the  subjects. 


Figure  2.  Pilot  #2  - Simple  Screen  Shot 
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Figure  3.  Pilot#2  - Complex  Screen  Shot 
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After  the  study  was  run  and  the  groups  of  students  (control  n=  47,  programmed  instruction  n = 47, 
programmed  instruction  + animation  n = 44)  data  analyzed  five  items  from  the  drawing  test,  nine  items  from  the 
terminology  test,  and  ten  items  from  the  comprehension  test  and  zero  items  on  the  identification  test  were  identified 
with  a difficultly  of  .60  or  less  and  the  animation  developed  for  the  current  study 

Current  Study:  In  this  iteration  of  the  study  there  are  three  levels  of  animations  explored.  The  complex 
treatment  from  pilot  study  number  2 with  the  placed  animation  only  at  the  right  of  center  text  area  served  as  the  base 
animation  or  control  group.  The  second  treatment  (figure  4)  used  the  same  base  animation  as  the  control  but  added 
another  effect  of  a simple  reveal  and  fade  that  cycled  two  times  than  stayed  on  the  screen  until  the  subject  moved  on. 
The  third  treatment  (figure  5)  used  the  same  base  animation  as  the  control  but  added  another  effect  of  a progressive 
reveal  and  fade  that  cycled  two  times  than  stayed  on  the  screen  until  the  subject  moved  on.  The  belief  is  that  the 


Figure  4.  Current  Study  - Simple  Screen  Shot 
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Figure  5.  Current  Study  - Complex  Screen  Shot 
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forced  information  processing  through  the  use  of  progressive  display  of  information  will  not  only  be  more  effective 
in  directing  attention  but  also  establish  links  in  memory  that  the  simple  and  control  treatments  may  not. 

In  the  complex  or  progressive  reveal  treatment  the  subject  was  presented  three  units  (a  word,  or  a symbol,  or  a group 
of  words)  of  information  in  succession.  The  simple  reveal  treatment  had  the  same  information  but  it  was  displayed 
and  faded  in  all  at  once.  Additionally,  in  the  progressive  reveal  treatment  the  part  of  the  heart  or  first  part  of  the 
statement  faded  in,  then  in  the  on  screen  animation  the  corresponding  part  blinked  three  times,  then  the  next  two 
sections  of  the  phrase  or  idea  faded  in  after  a short  delay.  For  each  thought  or  idea  the  process  was  repeated  once 
for  a forced  display  of  the  information  happening  twice  on  any  given  screen.  At  the  conclusion  of  this  animated 
sequence  the  subject  could  choose  to  view  the  entire  animation  again  or  move  on  to  the  next  webpage.  It  should  be 
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noted  that  the  complex  animations,  by  the  very  nature  of  the  progressive  reveal,  required  more  time  to  display  and 
averaged  around  22  seconds  run  time  for  the  base  animation  and  the  progressive  reveal.  In  the  simple  treatments 
where  the  idea  was  revealed  all  at  once  after  the  base  animation  the  average  run  time  was  around  15  seconds.  In  the 
control  group  where  only  the  base  animation  was  displayed  the  average  was  around  11  seconds. 

The  subjects  were  given  a moderate  level  of  user  control,  they  could  replay  or  view  previous  animations  at 
any  time  during  the  instruction,  but  were  required  to  view  the  animation  on  a screen  before  being  allowed  to 
progress  through  the  lesson.  Additionally  the  environment  allowed  that  subjects  to  pull  up  static  images  of  all  the 
parts  of  human  heart  after  they  had  been  covered  in  the  instructional  unit.  There  was  a degree  of  linearity  in 
programmed  instructional  part  of  the  materials  as  there  were  required  to  correctly  answer  4 out  of  6 questions  on  the 
quizzes  or  they  were  automatically  sent  back  the  beginning  of  that  section,  generally  3 pages  prior.  If  an  acceptable 
score  was  achieved  on  the  practice  quizzes  they  were  able  to  progress  forward  normally. 

Dependent  Measures 

In  the  pilot  studies  and  in  this  iteration  of  the  study  four  criterion  measures  developed  by  Dwyer  (1965) 
each  consisting  of  20-items  was  employed  to  assess  the  participant’s  achievement.  It  should  be  noted  that  with  the 
exception  of  the  drawing  test,  which  was  administered  in  a paper  pencil  format,  all  other  assessments  and  quizzes 
were  administered  in  an  online  format  to  minimize  any  mixed  medium  effects  between  the  lesson  and  the 
assessment.  Reported  reliability  coefficients  (KR-20)  and  brief  explanation  adapted  from  Dwyer  (1978,  pp.  45-47) 
of  each  tool  are  outlined  below: 

Drawing  Test.  (K-R  20:  .91)  A 20  object  queued  recall  test  designed  to  evaluate  student  ability  to  construct 
and/or  reproduce  items  in  their  appropriate  context.  The  drawing  test  provided  the  students  with  a numbered  list  of 
terms  corresponding  to  the  parts  of  the  heart  discussed  in  the  instructional  presentation.  The  students  were  required 
to  draw  a representative  diagram  of  the  heart  and  place  the  numbers  of  the  listed  parts  in  their  respective  positions. 
For  this  test,  the  emphasis  was  on  the  correct  positioning  of  the  verbal  symbols  with  respect  to  one  another  and  in 
respect  to  their  concrete  referents.  Conceptual  level  educational  objectives  were  addressed  with  this  assessment. 

Identification  Test.  (K-R  20:  .86)  A 20  question  - 5 option  multiple  choice  test  designed  to  evaluate  student 
ability  to  identify  parts  or  positions  of  an  object  in  the  heart.  This  multiple-choice  test  required  students  to  identify 
the  numbered  parts  on  a supplied  detailed  drawing  of  a heart.  Each  part  of  the  heart,  which  had  been  discussed  in 
the  presentation,  was  numbered  on  the  drawing.  The  objective  of  this  test  was  to  measure  the  ability  of  the  student 
to  use  visual  cues  to  discriminate  one  structure  of  the  heart  from  another  and  to  associate  specific  parts  of  the  heart 
with  their  proper  names.  Factual  level  educational  objectives  were  addressed  with  this  assessment. 

Terminology  Test.  (K-R  20:  .87)  A 20  question  - 5 option  multiple  choice  test  designed  to  measure 
knowledge  of  specific  facts,  terms,  and  definitions.  This  multiple-choice  test  consisted  of  items  designed  to  measure 
knowledge  of  specific  facts,  terms,  and  definitions.  The  objectives  measured  by  this  type  of  test  are  appropriate  to 
all  content  areas  that  have  an  understanding  of  the  basic  elements  as  a prerequisite  to  the  learning  of  concepts,  rules, 
and  principles.  Specifically,  conceptual  level  educational  objectives  were  addressed  with  this  assessment  as  a spring 
board  for  high  level  objectives. 

Comprehension  Test.  (K-R  20:  .84)  A 20  question  - 4 option  multiple  choice  test  designed  to  measure  a 
type  of  understanding  in  which  the  individual  can  use  the  information  being  received  to  explain  some  other 
phenomenon.  This  multiple-choice  test  consisted  of  items  where  given  the  location  of  certain  parts  of  the  heart  at  a 
particular  moment  of  the  heart  beat  cycle,  the  student  was  asked  to  determine  the  position  of  other  specified  parts  in 
the  heart  at  the  same  time.  This  test  required  that  the  students  have  a thorough  understanding  of  the  heart,  its  parts, 
its  internal  functioning,  and  the  simultaneous  processes  occurring  during  the  systolic  and  diastolic  phases  of  the 
heart  beat.  The  comprehension  test  was  designed  to  measure  a type  of  understanding  in  which  the  individual  can 
use  the  information  being  received  to  explain  some  other  phenomenon  and  addressed  the  rule  / procedural 
knowledge  educational  objectives. 

Total  Criterion  Score.  (K-R  20:  .95).  All  items  contained  in  the  previously  noted  individual  tests  were 
combined  into  a composite  test  score  to  measure  overall  learning  and  understanding. 

Validity  of  Dependent  Measures 

Upon  examination  of  the  content  pages,  instructional  script  and  materials  face  validity  is  believed  to  be 
evident  by  the  researchers.  Given  that  the  tests  were  textually  identical  in  the  web  based  version  used  in  this  study 
content  and  construct  validity  for  dependent  measures  is  assumed  as  part  of  the  original  instruments  umbrella.  An 
explanation  of  the  process  used  in  establishing  validity  is  covered  in  detail  greater  in  Dwyer  1978  and  in  his  1965 
thesis.  In  short  the  instructional  materials  and  dependent  measures  were  put  through  content  expert  and  educational 
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expert  review  with  objectives  and  descriptions  were  developed  and  employed  during  test  development  phase  by 
Dwyer  originally  in  1965  as  part  of  his  doctoral  thesis. 


Results 

For  statistical  analysis  the  alpha  level  was  set  a priori  at  the  0.05  level,  and  an  ANOVA  was  conducted  on 
each  dependent  measure.  Where  significant  differences  are  obtained,  Scheffe  or  Dunnett  C,  according  to  Levene’s 
test  of  homogeneity  of  variance,  follow  -up  procedures  were  implemented.  From  the  outset  of  the  study  it  was  the 
intention  of  the  researchers  to  analyze  the  data  from  each  criterion  measure  individually  and  collectively  and  in  an 
effort  to  comprehensively  examine  the  effects  the  animation.  In  the  table  1 below  the  descriptive  statistics  for  all 
items  are  displayed. 


Table  1.  Means  and  Standard  Deviations  for  Each  Treatment  on  Each  Dependent  Measure 
Dependent  Measure  Base  Animation  Base  Animation  with  Base  Animation  with 

Mean  # Correct  Control  Group  (n=29)  Reveal  (n=31)  Progressive  Reveal  (n=28) 

Standard  Deviation. 


Drawing 


Mean 

15.97 

16.00 

18.07 

S.D. 

3.26 

3.61 

2.26 

Identification 

Mean 

17.62 

18.35 

18.45 

S.D. 

2.47 

1.70 

1.99 

Terminology 

Mean 

11.93 

13.65 

15.10 

S.D. 

5.03 

3.96 

3.30 

Comprehension 

Mean 

11.00 

12.55 

12.79 

S.D. 

3.64 

3.67 

3.51 

Total  Criterion  Score 
Mean 

56.52 

45.55 

64.75 

S.D. 

11.90 

7.76 

8.75 
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Tables  2 and  3 show  the  results  of  the  One-Way  Anova  and  the 
appropriate  follow  up  tests  used.  Significant  effects  were  found  on 
the  Drawing  (F=4.29  df  (2,85),  p=.02)  and  Terminology  (F=4.25  df 
(2,86),  p=.02)  and  Total  Criterion  (F=29.767  df  (2,85),  p<.001).  In 
the  case  of  the  drawing  test,  after  using  Scheffe  post  hoc  test  it  was 
determined  that  the  progressive  reveal  group  did  significantly  better 
than  both  the  simple  reveal  and  control  groups.  For  the  Terminology 
test,  after  using  Dunnett  C follow  up  test  because  it  failed  the  Levene 
test  of  homogeneity  it  was  determined  that  the  progressive  reveal 
group  statistically  performed  significantly  better  than  the  control 
group  with  base  animation  only.  When  looking  at  the  Total  Criterion 
measure  the  assumption  of  Levene’ s test  of  homogeneity  test  was  not 
supported  and  Dunnett  C was  implemented  on  the  results.  The 
results  were  statistically  significant  in  all  comparisons  and  best 
illustrated  by  Figure  6. 


Figure  6.  Total  Criterion  Means  Plot 


Table  2.  Results  of  Analysis  of  Variance  Across 

Sums  of 
Squares 

All  Items 

Mean 

df  Square 

F 

sig 

Drawing 

Between  groups 

83.26 

2 

41.63 

4.29 

.016 

Within  Groups 

824.82 

85 

9.70 

Total 

908.08 

87 

Identification 

Between  groups 

12.00 

2 

6.00 

1.40 

.253 

Within  Groups 

369.10 

86 

4.29 

Total 

381.10 

88 

Terminology 

Between  groups 

146.26 

2 

73.13 

4.25 

.017 

Within  Groups 

1481.65 

86 

17.23 

Total 

1627.91 

88 

Comprehension 

Between  groups 

55.20 

2 

27.60 

2.189 

.118 

Within  Groups 

1084.44 

86 

12.61 

Total 

1139.64 

88 

TOTAL 

Between  groups 

5491.29 

2 

2745.64 

29.77 

.000 

Within  Groups 

7840.17 

85 

92.234 

Total 

13331.46 

87 

Table  3.  Analysis  of  Variance  Follow 

Up  Tests  With  Significance 

All  Items 

Test 

Follow  Up  Used 
Levene  Statistic  Sig. 

Group  1 

Group  2 

Mean 

Difference 

Drawing 

Scheffe 

Complex 

Control 

2.11 

.13 

Simple 

2.07 

Terminology 

Dunnett  C 

Complex 

Control 

3.17 

.001 

TOTAL 

Dunnett  C 

Complex 

Simple 

8.23 

.010 

Control 

19.20 

Control 

Simple 

10.97 

Also  of  interest  was  the  analysis  run  on  just  the  items  addressed  in  the  treatments.  For  the  Drawing  test 
there  were  five  items  that  had  and  item  difficulty  less  than  .60  after  an  item  analysis.  Similarly  the  Terminology  and 
Comprehension  tests  had  nine  and  ten  items  respectively  that  were  specifically  targeted.  Table  4 displays  the 
descriptive  statistics  for  the  items  addressed.  Table  5 shows  the  results  of  the  analysis  of  variance  and  table  six 
displays  the  appropriate  follow-up  tests  and  their  significant  results.  It  should  be  noted  that  there  were  no  items  on 
the  identification  test  with  a difficulty  level  less  than  .60  and  therefore  were  not  addressed.  Also  the  terminology 
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test  was  found  significant  with  comparing  all  20  items  on  the  test  but  when  looking  at  the  9 items  addressed 
specifically  it  just  misses  the  .05  alpha  level  coming  in  with  p =.075. 


Table  4.  Means  and  Standard  Deviations  for  Individual  Items  Addressed  on  Each  Dependent  Measure 
Dependent  Measure  (#  of  Base  Animation  Base  Animation  with  Base  Animation  with 

items  address)  Control  Group  (n=29)  Reveal  (n=31)  Progressive  Reveal  (n=28) 

Mean  # Correct 
Standard  Deviation. 


Drawing  (5  Items) 


Mean 

3.28 

3.81 

4.50 

S.D. 

1.60 

1.28 

0.73 

Identification  (0  Items) 

N/A 

N/A 

N/A 

Terminology  (9  Items) 
Mean 

4.14 

4.77 

5.52 

S.D. 

2.49 

2.29 

2.01 

Comprehension  (10  Items) 
Mean 

5.14 

5.06 

5.21 

S.D. 

1.66 

5.21 

1.78 

Total  Score 
Mean 

12.55 

13.65 

15.32 

S.D. 

4.56 

3.63 

3.61 

Table  5.  Results  of  Analysis  of  Variance  Addressed  Items  Only 


Sums  of 
Squares 

Df 

Mean 

Square 

F 

Sig 

Drawing 

Between  groups 

21.45 

2 

10.72 

6.62 

.002 

Within  Groups 

137.63 

85 

1.62 

Total 

159.08 

87 

Identification 

None  Targeted 

Terminology 

Between  groups 

27.64 

2 

13.82 

2.68 

.075 

Within  Groups 

444.11 

86 

5.16 

Total 

471.75 

88 

Comprehension 

Between  groups 

0.31 

2 

0.15 

0.05 

.951 

Within  Groups 

258.38 

86 

3.00 

Total 

88 

TOTAF 

Between  groups 

110.72 

2 

55.36 

3.53 

.034 

Within  Groups 

1332.38 

85 

15.68 

Total 

1443.09 

87 

Table  6.  Analysis  of  Variance  Follow  Up  Tests  With  Significance 
Addressed  Items  Only 


Test 

Follow  Up  Used 
Fevene  Statistic  Sig. 

Group  1 

Group  2 

Mean 

Difference 

Drawing 

Dunnett  C 
.004 

Complex 

Control 

1.22 

TOTAF 

Scheffe 

.246 

Complex 

Control 

2.77 

Discussion 

From  the  initial  pilot  study  run  in  the  fall  of  2003  there  has  been  marked  improvement  in  the  raw  mean 
scores  with  each  successive  iteration  beyond  simple  static  visuals,  first  using  programmed  instruction,  and  then 
adding  in  basic  animation,  and  finally  with  the  additional  animation  strategy  of  simple  reveal  vs.  progressive  reveal. 
Results  of  this  study  have  shown  some  statistically  significant  findings  for  some  educational  variable  levels  and  on 
the  whole  but  not  for  all  educational  levels. 
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The  results  of  this  study  indicate  that  the  use  of  animation  when  properly  designed  and  positioned  is  an 
important  instructional  variable  for  complementing  web-based  instruction.  However,  it  should  not  be  seen  as  a 
panacea  that  can  cure  all  the  ills  of  instruction  and  education.  Results  have  also  shown  that  the  use  of  animation  is 
not  equally  effective  for  facilitating  achievement  across  all  of  the  different  levels  of  educational  objectives.  Even 
when  subjects  have  the  perquisite  knowledge  required  to  build  upon  for  success  at  the  higher  levels  achievement  is 
not  guaranteed.  In  the  case  of  this  study  the  researcher  went  to  great  pains  to  ensure  that  lower  level  objectives 
where  addressed  in  sufficient  detail  and  we  believe  that  we  succeeded  with  the  control  group  averaging  almost  80 
and  90  percent  on  the  drawing  and  identification  tests  respectively. 

One  interesting  note  was  the  large  drop  in  the  overall  and  addressed  items  mean  scores  exhibited  by  the 
simple  reveal  group.  Possible  explanations  of  this  apparent  anomaly  vary  and  were  a frequent  topic  of  discussion  by 
the  researchers.  One  suggestion  is  that  of  a statistical  anomaly.  We  randomly  assigned  all  subjects  to  treatments  but 
there  is  still  a chance  that  it  was  a random  error.  We  are  fairly  certain  that  we  have  avoided  common  pitfalls  in  our 
research  design  but  a replication  study  is  currently  planned  for  the  spring  of  2005  and  this  competing  explanation 
can  explored  in  more  detail  then.  Another  possible  explanation  discussed  by  the  group  was  lack  of  motivation  by 
the  individuals  randomly  assigned  to  this  treatment,  but  once  again  we  used  random  assignment  this  should  have 
distributed  this  equally  across  groups.  Further,  upon  inspection  of  the  data  and  answers  keyed  into  the  assessments 
there  did  not  appear  to  be  any  obvious  patterns  or  clues  indicating  an  obvious  lack  of  effort  by  students.  Participants 
were  also  allowed  to  exit  the  study  at  anytime  if  they  so  chose.  No  one  exercised  this  option  but  all  subjects  were 
made  aware  of  it  at  the  onset  when  they  read  and  signed  their  informed  consent  forms. 

One  possibility  that  seemed  to  resonate  with  researchers  dealt  with  the  level  of  cognitive  processing  load, 
part  of  the  underlying  theoretical  basis  for  the  study,  was  exceeded.  Even  though  the  subjects  in  the  simple  reveal 
and  progressive  viewed  the  same  content  and  all  textual  and  graphical  information  was  identical,  the  presentation  of 
information  in  the  simple  reveal  increased  the  cognitive  processing  load  beyond  their  individual  thresholds.  It  is 
suggested  that  the  subjects  could  no  longer  focus  and  direct  attention  and  that  onslaught  of  information  that  was 
taken  in  for  processing  was  quickly  disposed  of.  While  experiencing  this  process  of  overload  the  subjects  actually 
missed  the  other  bits  of  information  presented  in  all  treatments  that  the  control  and  complex  groups  were  able  to 
attune  to.  In  future  studies  qualitative  data  and  questions  asking  the  students  to  rate  the  data  presentation  may  be 
included  but  make  the  assumption  that  the  subject  is  aware  of  the  overload,  which  may  or  may  not  be  the  case. 

Another  interesting  point  that  needs  to  be  explored  further  is  that  when  all  items  in  the  comprehension 
dependent  measure  the  analysis  of  variance  returned  an  F-ratio  of  2.19,  p=.12  but  when  compared  to  items  addressed 
the  F-ratio  is  .05  and  p=.951.  When  viewed  in  tandem  it  appears  that  most  if  not  all  of  the  variance  increase  was  on 
the  ten  out  of  twenty  items  that  weren’t  specifically  targeted  or  addressed  by  treatments.  Further  analysis  is  required 
to  address  this  question. 

What  remains  to  be  seen  is  the  transfer  and  replication  of  these  findings  in  future  studies.  Transfer  of  the 
findings  of  this  study  can  be  logically  extended  other  domain  areas  where  information  is  organized  in  a hierarchal 
manner.  However,  widespread  generalization  of  findings  is  a dangerous  proposition;  never-the-less,  implications  for 
practitioners  can  be  generated  so  long  as  they  are  ‘taken  with  a grain  of  salt’  and  not  assumed  to  be  fact  but  merely  a 
possible  lens  from  which  to  view  a problem.  The  use  of  animation  increased  overall  achievement  on  the  parts  of  the 
learners  due  in  large  part  to  effects  on  the  factual  and  conceptual  levels.  Given  the  cost  of  development  for  use  of 
animation  this  should  be  considered  within  a contextual  frame  work  of  what  is  it  worth  to  you?  If  similar  findings 
can  be  achieved  with  well  designed  and  systematically  placed  static  visuals  the  benefits  out  weigh  the  costs.  There 
were  clear  areas  where  progressive  reveal  animation  strategies  were  better  in  facilitating  achievement.  This  relates 
to  the  clarity  of  the  relationship  between  the  way  the  animation  was  designed  and  information  presented.  When 
choosing  to  utilize  animation,  specific  attention  needs  to  be  devoted  to  the  cognitive  load  on  the  individual.  If  items 
require  too  great  a processing  time  than  they  can  do  more  detriment  than  not  using  them  at  all  in  an  instructional 
module. 
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Abstract 

This  paper  considers  four  aspects  of  online  communities.  Design,  mechanisms,  architecture,  and  the 
constructed  knowledge.  We  hypothesize  that  different  designs  of  communities  drive  different  mechanisms,  which  give 
rise  to  different  architectures,  which  in  turn  result  in  different  levels  of  collaborative  knowledge  construction.  To 
test  this  chain  of  hypotheses,  we  analyzed  the  recorded  responsiveness  data  of  two  online  communities  of  learners 
having  different  designs:  a formal,  structured  team,  and  an  informal,  non- structured,  Q&A  forum.  The  designs  are 
evaluated  according  to  the  Social  Interdependence  Theory  of  Cooperative  Learning.  Knowledge  construction  is 
assessed  through  Content  Analysis.  The  architectures  are  revealed  by  Statistical  Analysis  ofp*  Markov  Models  for 
the  communities.  The  mechanisms  are  then  identified  by  matching  the  predictions  of  Network  Emergence  Theories 
with  the  observed  architectures.  The  hypotheses  are  strongly  supported.  Our  analysis  shows  that  the  minimal- effort 
hunt-for-social-capital  mechanism  controls  a major  behavior  of  both  communities:  negative  tendency  to  respond. 
Differences  in  the  goals,  interdependence  and  the  promotive  interaction  features  of  the  designs  of  the  two 
communities  lead  to  the  development  of  different  mechanisms:  cognition  balance  and  peer  pressure  in  the  team,  but 
not  in  the  forum.  Exchange  mechanism  in  the  forum,  but  not  in  the  team.  In  addition,  the  pre-assigned  role  of  the 
tutor  in  the  forum  gave  rise  to  its  responsibility  mechanism  in  that  community,  but  not  in  team  community.  These 
differences  in  the  mechanisms  led  to  the  formation  of  different  sets  of  virtual  neighborhoods,  which  show  up 
macro scopically  as  differences  in  the  cohesion  and  the  distribution  of  response  power.  These  differences  are 
associated  with  the  differences  in  the  buildup  of  knowledge  in  the  two  communities.  The  methods  can  be  extended  to 
other  relations  in  online  communities  and  longitudinal  analysis,  and  for  real-time  monitoring  of  online 
communications. 


Introduction 

Building  communities  is  recognized  as  an  essential  strategy  for  online  learning.  An  online  community 
consists  of  actors  who  develop  certain  relations  among  themselves.  For  example,  some  actors  only  read  what  others 
write;  some  respond  to  queries  posted  by  others  and  some  influence  others  to  do  something  (for  example  to  access  a 
web  page),  etc.  This  simple  observation  led  us  to  adopt  a network  abstraction  to  describe  online  communities.  A 
network  is  a set  of  actors  - members  of  communities,  groups,  web-pages,  countries,  genes,  etc.,  with  certain  possible 
relations  between  pairs  of  actors.  The  relations  may  - or  may  not  - be  hierarchical,  symmetrical,  binary,  or  other. 
Network  abstraction  is  thus  very  flexible. 

Social  Network  Analysis  (Wasserman  and  Faust  1999)  is  a useful  tool  for  studying  relations  in  a network. 

It  is  a collection  of  graph  analysis  methods  developed  by  researchers  to  analyze  networks  which  consist  of  precise 
mathematical  definitions  of  certain  network  structures  and  the  methods  to  calculate  them.  Examples  of  network 
structures  are  cohesiveness  and  transitivity:  cohesiveness  measures  the  tendency  to  form  groups  of  strongly 
interconnected  actors;  transitivity  measures  the  tendency  to  form  transitive  triad  relations  (if  i relates  to  j and  j 
relates  to  k,  then  i necessarily  also  relates  to  k).  SNA  has  been  utilized  to  analyze  networks  in  various  areas,  whose 
actors  include  politicians  (Faust,  Willet  et  al.  2002),  the  military  (Dekker  2002),  adolescents  (Ellen,  Dolcini  et  al. 
2001),  multi-national  corporations  (Athanassiou  1999),  families  (Widmer  and  La  Farga  1999),  and  terrorist 
networks  (van  Meter  2002).  SNA  methods  were  introduced  to  online  communities  research  in  (Garton, 
Haythornthwaite  et  al.  1997).  Since  then  several  scholars  have  demonstrated  the  applicability  of  SNA  to  specific 
collaborative  learning  situations  (Haythornthwaite  1998;  Wortham  1999;  Lipponen,  Rahikainen  et  al.  2001;  Cho, 
Stefanone  et  al.  2002;  de  Laat  2002;  Martinez,  Dimitriadis  et  al.  2002;  Reffay  and  Chanier  2002;  Aviv  2003). 

Macro-level  SNA  identifies  network  macro- structures  such  as  cohesiveness.  Micro  level  SNA  reveals 
significant  underlying  microstructures,  or  neighborhoods,  such  as  transitive  triads  (Pattison  and  Robbins  2000; 
Pattison  and  Robbins  2002).  The  identified  neighborhoods  are  the  basis  for  revealing  theories  that  explain  their 
emergence  (Contractor,  Wasserman  et  al.  1999).  For  example,  the  theory  of  cognitive  balance  explains  the 
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emergence  of  transitive  triads,  which  underlies  the  macroscopic  phenomenon  of  cohesiveness.  The  precise  definition 
of  a neighborhood  will  be  given  in  section  2. 

We  examine  online  communities  of  learners  according  to  the  constructivist  perspective  (Jonassen, 

Davidson  et  al.  1995).  Rafaeli  (Rafaeli  1988)  emphasized  that  constructive  communication  is  determined  by  its 
responsiveness.  Accordingly,  we  analyze  the  network  structures  of  the  responsiveness  relation  between  actors  in  the 
online  communities.  Previous  work  (Aviv,  Erlich  et  al.  2003)  demonstrated  that  certain  macrostructures  (cohesion, 
centrality  and  role  groups)  are  correlated  with  the  design  of  the  communities  and  with  the  quality  of  the  constructed 
shared  knowledge.  In  this  study,  we  extract  the  micro-level  neighborhoods  of  the  same  communities.  Our  goal  is  to 
reveal  the  underlying  theoretical  mechanisms  that  control  the  dynamics  of  the  communities  and  to  correlate  them 
with  the  design  parameters  and  with  the  quality  of  the  knowledge  constructed  by  the  communities. 

Architecture  of  a Community 

An  online  community  is  modeled  as  a network  of  actors.  Every  ordered  pair  of  actors  has  a potential 
response  tie  relation.  The  response  tie  between  actor  i and  actor  j is  realized  if  i responded  to  at  least  one  message 
sent  by  j to  the  community;  otherwise  the  response  tie  is  not  realized.  In  addition,  a (non-directed)  viewing  relation 
is  realized  between  a pair  of  actors  if  they  read  the  same  messages.  In  a broadcast  community,  a realized  response  tie 
relation  is  also  a realized  viewing  tie.  The  reverse  is  not  necessarily  true. 

A virtual  neighborhood  (VN)  is  a sub-set  of  actors,  endowed  with  a set  of  prescribed  possible  response  ties 
between  them,  all  of  which  are  pair-wise  statistically  dependent.  We  identified  the  significant  VNs  of  a community 
by  fitting  a p*  stochastic  Markov  model  (Wasserman  and  Pattison  1996;  Robins  and  Pattison  2002)  to  the  response 
tie  data.  In  this  model,  every  pair  of  response  ties  in  a VN  has  a common  actor,  which  is  why  they  are 
interdependent.  Same  topology  VNs  are  aggregated  into  a class  of  VNs.  In  the  model  every  possible  class  is 
associated  with  a strength  parameter  that  measures  the  tendency  of  the  network  to  realize  VNs  of  that  class.  The 
basic  ideas  and  the  formulas  of  the  p*  Markov  model  are  elaborated  in  (Wasserman  and  Pattison  1996;  Robins  and 
Pattison  2002).  The  model  equations  are  presented  in  the  Appendix.  Examples  of  Markov  VNs  are  presented 
graphically  in  Figure  1 . 


In  this  research  we  consider  the  set  of  Markov  classes  of  VNs  listed  in  Table  1. 


VN  Class 

Participating  Actors  & Prescribed  Response 1 

link 

All  pairs:  (i— q)  or  (j  — >i) 

respi 

All  pairs:  (i— q)  fixed  i 

triggi 

All  pairs:  (j— d)  fixed  i 

mutuality 

All  pairs:  (i— q)  and  (j  — d) 

out- stars 

All  triplets:  (i— q)  and  (i— >k) 

in-stars 

All  triplets:  (i— q)  and  (k— q) 

mixed- stars 

All  triplets:  (i— q)  and  (j— >k) 

transitivity 

All  triplets:  (i— q)and(j— >k)and  (i— >k) 

cyclicity 

All  triplets:  (i— q)  and  (j — >k)  and  (k— d) 

Table  1.  Classes  of  VNs 
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Tendencies  to  form  VNs  of  a certain  class  are  the  result  of  the  underlying  mechanisms.  Several  candidate 
mechanisms,  postulated  by  certain  network  emergence  theories  are  briefly  described  below.  See  (Monge  and 
Contractor  2003)  for  an  extensive  survey. 

The  theory  of  social  capital  (Burt  1992;  Burt  2002)  postulates  efficient  connectivity  in  the  hunt  for  a social 
capital  mechanism.  In  an  online  broadcast  community,  efficiency  means  forming  zero  response  ties  because  a 
response  tie  is  a redundant  viewing  tie,  so  actors  prefer  to  remain  passive.  This  mechanism  predicts  a tendency  for 
not  creating  VNs  of  any  class.  Thus,  other  mechanisms  are  responsible  for  creating  responsiveness. 

Exchange  and  resource  dependency  theories  (Homans  1958;  Wilier  1999)  postulate  an  information 
exchange  mechanism,  in  which  actors  prefer  to  forge  ties  with  potentially  “resource-promising”  peers.  This 
mechanism  creates  tendency  for  VNs  of  class  mutuality. 

The  theory  of  generalized  exchange  (Bearman  1997)  postulates  an  information  exchange  mechanism  via 
mediators.  This  theory  then  predicts  tendencies  for  n-link  cycles,  in  particular  VNs  from  the  cyclicity  class. 

Theories  of  collective  action  (Marwell  and  Oliver  1993)  postulate  a social  pressure  mechanism  that  induces 
actors  to  contribute  to  the  goal  of  the  community  if  threshold  values  of  “pressing”  peers,  existing  ties,  and  central 
actors  are  met  (Granovetter  1983;  Valente  1996).  In  that  case,  actors  will  respond  to  several  others,  forging  out-stars 
VNs. 

Contagion  theories  (Burt  1987;  Contractor  and  Eisenberg  1990)  postulate  that  the  exposure  of  actors  leads 
to  a contagion  mechanism  that  uses  social  influence  and  imitation  to  create  groups  of  equivalent  actors  with  similar 
behaviors  (Carley  and  Kaufer  1993).  Contagion  predicts  a tendency  for  VNs  of  the  various  star  shaped  classes. 


Theories 
Social  capital 
Collective  action 

Exchange 

Generalized  exchange 
Contagion 

Cognitive  consistency 
Uncertainty  reduction 
Exogenous  factors: 

Students 

Exogenous  factors:  Tutors 
Table  2:  Research  Hypotheses 


Predicted  Tendencies 
Few  single  tie  links 
If  thresholds  met  then  respond 
to  several  others 
Tendency  to  reciprocate 
Tendency  to  respond  cyclically 
Respond  to  same  as  others 
Respond  via  several  paths 
Attract  many  responses 
No  tendencies  to  respond/trigger 

Personal  tendencies  to 
respond/trigger 


Hypotheses 
HI:  link  < 0 

H2:  if  thresholds  met  then  out- stars  > 0 

H3:  mutuality  > 0 
H4:  cyclicity  > 0 

H5:  out-stars>  0;  in-stars  > 0;  mixed-stars  > 0 

H6:  transitivity  > 0 

H7:  in-stars  > 0 

H8:  {respi  = 0 I i C students} 

H9:  { triggi  = 0 I i C students} 

H10:  {respi  > 0 I i = tutor} 

HI  1 : { triggi  > 0 I i = tutor} 


Theories  of  cognitive  balance  (Cartwright  and  Harary  1956;  Festinger  1957;  Harary,  Norman  et  al.  1965) 
postulate  a cognition  balance  mechanism  with  a drive  to  overcome  dissonance  and  achieve  cognition  consistency 
among  actors.  This  drive  is  implemented  by  transitivity  VNs. 

The  uncertainty  reduction  theory  (Berger  1987)  postulates  drives  in  actors  to  forge  links  with  many  others 
to  reduce  the  gap  of  the  unknown  between  themselves  and  their  environment;  this  theory  predicts  a tendency  to 
create  in-stars  (responses  to  triggering  actors)  VNs. 

Finally,  responsibilities  of  actors  influence  their  residual  personal  tendencies  toward  response  ties.  In  this 
study,  students  did  not  have  pre-assigned  responsibilities,  predicting  that  the  students’  VNs  respi  and  triggi  will  be 
insignificant.  The  tutors’  residual  tendencies  will  be  significant,  due  to  their  roles. 

The  theories,  and  predicted  tendencies  stated  as  Research  Hypotheses,  are  presented  in  Table  2. 


The  Analysis 

We  analyzed  recorded  transcripts  of  two  online  communities  - two  communities  of  students  at  the  Open 
University  of  Israel.  These  communities  were  established  for  17  weeks  during  the  Fall  2000  semester  (19 
participants)  and  the  Spring  2002  semester  (18  participants)  as  part  of  an  academic  course  in  Business  Ethics.  Each 
community  included  one  tutor.  The  designs  of  the  activities  of  the  two  communities  were  different.  The  Fall  2000 
community  was  designed  as  a goal-directed  collaborative  team,  whereas  the  Spring  2002  community  was  a Q&A 
forum.  Hence  we  have  labeled  the  communities  “team”  and  “forum,”  respectively.  The  data  is  available  at 
http://telem.openu.ac.il/courses  (password  protected). 

The  team  community  engaged  in  a formal  debate.  Participants  registered  and  committed  to  active 
participation,  with  associated  rewards  in  place.  Students  took  the  role  of  an  "advisory  committee"  that  had  to  advise 
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a company  on  how  to  handle  the  business/ethical  problem  of  cellular  phone  emissions.  The  debate  was  scheduled  as 
a 5-step  process  of  moral  decision-making,  with  predefined  goals  (Geva  2000).  A unique  feature  of  the  team 
community  was  that  the  goals  of  the  debate  were  to  reach  consensus  up  to  the  point  of  writing  a joint  proposal  to  an 
external  agency.  The  forum  community  was  open  to  all  students  in  the  course.  Participants  were  asked  to  raise 
questions  on  issues  relating  to  the  course.  We  followed  the  social  interdependence  theory  of  cooperative  learning 
(Johnson  and  Johnson  1999)  to  characterize  the  communities  according  to  four  groups  of  parameters: 
interdependence,  promotive  interaction,  pre-assigned  roles,  and  reflection.  The  two  communities  differ  in  most  of 
the  design  parameters.  Table  3 summarizes  the  differences  between  the  designs  of  the  two  communities. 


Parameter 

Team 

Forum 

Registration  & commitment 

Yes 

No 

Interdependence:  deliverables 

Yes 

No 

Interdependence:  tasks  & schedule 

Yes 

No 

Interdependence:  resources 

Yes 

No 

Reward  mechanism 

Yes 

No 

Interdependence:  reward 

No 

No 

Promotive  interaction:  support  & help 

Yes 

No 

Promotive  interaction:  feedback 

Yes 

No 

Promotive  interaction:  advocating  achievements 

No 

No 

Promotive  interaction:  monitoring 

Yes 

No 

Pre-assigned  roles:  tutor 

No 

Yes 

Pre-assigned  roles:  students 

No 

No 

Reflection  procedures 

No 

No 

Individual  accountability 

Yes 

No 

Social  skills 

Yes 

Yes 

Table  3:  Design  of  Communities 

Previous  analysis  (Aviv,  Erlich  et  al.  2003)  analyzed  the  constructed  knowledge  and  the  macro- structures 
of  the  communities.  The  analysis  revealed  that  the  team  community  exhibited  high  levels  of  constructing 
knowledge,  developed  a mesh  of  interlinked  cliques,  and  that  many  participants  took  on  bridging  and  triggering 
roles  while  the  tutor  remained  on  the  side.  The  forum  community  did  not  construct  knowledge,  cohesion  was  dull, 
and  only  the  tutor  had  a special  role.  In  the  team  community,  many  students  participate  in  many  cliques,  but  the  tutor 
belongs  to  only  one  clique.  In  the  forum  community,  only  one  student  and  the  tutor  belong  to  the  two  cliques  that 
developed.  In  addition,  participants  in  the  team  community  shared  the  role  of  responders  among  them,  whereas  in 
the  forum  community  only  the  tutor  was  a central  responder. 

The  p*  model  of  the  team  community  has  43  classes  of  virtual  neighborhoods,  each  with  its  explanatory 
and  parameter:  18  respi9  18  triggi,  link,  mutuality,  transitivity,  cyclicity,  and  the  three  stars.  Similarly,  the  model  of 
the  forum  community  includes  45  classes  of  virtual  neighborhoods:  19  respi?  19  triggi,  link,  mutuality,  transitivity, 
cyclicity,  and  the  three  stars.  The  explanatories  count  the  number  of  virtual  neighborhoods  that  were  completely 
realized  in  the  networks.  The  strength  parameters  represent  the  tendency  to  create  (or  not)  neighborhoods  from  the 
classes. 

The  analysis  of  the  p*  model  consists  of  two  steps:  In  the  first  step  we  calculate  the  explanatories.  This  was 
performed  using  the  PREPSTAR  program  (Anderson,  Wasserman  et  al.  1999).  The  second  step  involves  solving  the 
binary  logistic  regression  (equation  A5).  The  solution  provides  an  approximate  estimate  for  the  strength  parameters. 
This  step  was  performed  with  SPSS.  Details  are  provided  in  the  Appendix.  We  configured  the  SPSS  binary  logistic 
procedure  to  work  in  forward  steps,  adding  one  class  of  virtual  neighborhoods  (i.e.,  its  explanatory)  at  a time, 
according  to  its  significance,  where  significance  was  assessed  by  the  decrements  in  the  PLLD  (Pseudo  Log- 
Likelihood  Deviance).  The  analysis  stops  when  no  more  significant  explanatory  variables  can  be  identified. 

The  analysis  revealed  three  significant  classes  of  virtual  neighborhoods  for  the  team  community,  and  four 
significant  classes  of  virtual  neighborhoods  for  the  forum  community.  The  PLLD  estimates  of  the  strength 
parameters  are  presented  in  Table  4. 
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Class 

Team 

0K 

SE 

Wald 

P 

exp(OK) 

link 

3.1 

3 

.32 

97.5 

.000 

.043 

out- star 

.18 

.06 

9.6 

.002 

1.199 

transiti 

vity 

Forum 

.31 

.06 

23.9 

.000 

1.366 

link 

2.6 

.8 

10.29 

.001 

.076 

respi8 

6.1 

.12 

26.78 

.000 

456.28 

mutuali 

ty 

6.2 

1.38 

20.61 

.002 

519.92 

in-stars 

- 

.91 

12.39 

.000 

.041 

3.2 


Table  4:  Revealed  VNs 

In  Table  4,  0K  is  the  MPLE  (maximal  pseudo-likelihood  estimator)  for  the  strength  parameter  of  class  K of 
VNs;  SE  is  an  estimate  of  its  associated  standard  error,  exp(0K)  measures  the  increase  (or  decrease,  if  0K  negative) 
in  the  conditional  odds  of  creating  a response  tie  between  any  pair  of  participants  if  that  response  tie  completes  a 
new  VN  of  class  K. 

We  tested  the  hypotheses  that  OK  = 0 by  the  Wald  parameter  (0K/SE)2  which  is  assumed  to  have  chi  square 
distribution.  Table  3 shows  that  all  these  null  hypotheses  were  rejected  with  extremely  small  p values. 

The  statistical  distributions  of  the  MPLEs  and  the  Wald  parameters  are  unknown  (Robins  and  Pattison  2002),  so 
inferences  are  not  precise  in  the  pure  statistical  sense. 


Results 

Few  classes  of  VNs  are  significant:  3 in  the  Team,  4 in  the  Forum.  In  particular,  the  personal  classes  of 
VNs  of  students,  respi  and  triggi,  are  not  significant.  This  corroborates  hypotheses  H8  and  H9. 

The  relative  importance  of  the  classes  of  VNs  is  depicted  by  their  contributions  to  the  goodness  of  fit  of  the  Markov 
models.  These  are  presented  in  Figure  2. 


Figure  2.  Relative  importance  of  VNs 


Figure  2 shows  that  the  global  class  link  of  the  single  response  tie  virtual  neighborhoods  is  the  most 
significant  in  both  communities:  65%  and  72%  of  the  goodness  of  fit  are  explained  by  the  tendencies  associated  with 
this  class.  Table  5 shows  that  in  both  communities  the  strength  parameter  0 of  the  link  class  is  negative.  This  means 
that  the  major  observed  phenomenon  in  both  communities  is  a significant  tendency  for  not  creating  single  response 
tie  neighborhoods  - the  phenomenon  of  lurking.  As  elaborated  above,  this  can  be  explained  by  basic  self  interest  - 
minimizing  the  effort  required  to  forge  a response  tie  vs.  the  possible  social  capital  reward,  given  that  every 
response  tie  is  a redundant  viewing  tie.  This  observation  is  in  accordance  with  hypothesis  HI.  Note  that  the  fact  that 
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response  ties  are  redundant  viewing  ties  is  a feature  of  every  broadcast  community,  irrespective  of  the  design  of  the 
community. 

By  itself,  the  negative  tendency  to  create  virtual  neighborhoods  of  class  link  will  give  rise  to  a community 
of  non-responsive  isolates.  The  actual  responsiveness  is  formed  by  the  other  neighborhoods.  These  neighborhoods 
are  quite  different  in  the  two  communities.  The  significant  virtual  neighborhoods  in  the  team  community  are  from 
the  global  classes  transitivity  and  out-stars.  The  significant  virtual  neighborhoods  in  the  forum  community  are  from 
the  personal  class  respi8,  and  from  the  global  classes  mutuality  and  in-stars.  In  the  subsections  below,  we  will 
consider  each  of  these  virtual  neighborhoods. 

The  goodness  of  fit  of  the  Markov  model  for  the  team  community  increases  by  30%  when  the  transitivity 
class  of  virtual  neighborhoods  is  included.  In  this  community  there  is  a positive  tendency  to  create  transitive  virtual 
neighborhoods.  This  means  that  in  the  team  community,  the  likelihood  of  setting  up  a response  tie  from  any  actor  i 
to  any  other  actor  j is  enhanced  (by  1.37)  if  that  tie  completes  a transitive  triangle  virtual  neighborhood.  This  is 
relative  to  the  likelihood  of  setting  up  any  other  neighborhood.  No  such  preference  exists  in  the  forum  community. 
Hypothesis  H6  - the  tendency  for  creating  virtual  neighborhoods  of  the  transitivity  class  is  positive  - is  therefore 
accepted  for  the  team  community  but  rejected  for  the  forum  community. 

The  tendency  to  create  transitive  structures  can  be  explained  by  cognitive  balance  theory.  It  seems  that  the 
design  of  the  team  community  leads  to  the  cognition  balance  mechanism,  by  which  dissonance  between  actors  and 
between  their  perceptions  of  objects  is  resolved  by  balanced  paths  of  communication.  This  can  be  attributed  to  the 
interdependence  built  into  the  design  of  the  community  and  to  the  particular  goal  which  forced  the  participants  to 
reach  consensus  during  the  online  debate  (in  order  to  submit  joint  proposals).  The  forum  community,  on  the  other 
hand,  was  a series  of  typical  Q&A  sessions.  Here  each  of  the  students  participating  was  interested  at  a certain  point 
in  time  in  a specific  issue  usually  related  to  an  assignment.  The  scope  of  the  issue  was,  in  many  cases,  limited;  it 
interested  few  students.  Other  issues,  or  even  related  concepts  not  directly  connected  to  the  query,  were  less 
important  to  the  student  who  asked  the  question,  let  alone  to  other  students.  The  lifetime  of  each  issue  was  short 
(usually  until  the  assignment  due  date).  There  was  no  drive  to  settle  conceptual  inconsistencies  regarding  past  issues, 
or  dissonance  in  perceptions  regarding  others.  Thus,  no  cognition  balance  mechanism  was  needed  and  none  was 
established. 

Introducing  the  personal  class  respi8  to  the  model  of  the  forum  community  increases  its  goodness  of  fit  by 
21%.  This  class  includes  all  the  virtual  neighborhoods  of  single  response  ties  initiated  by  N1 8 - the  tutor.  This 
means  that  the  residual  tendency  of  N 1 8 to  respond  - above  and  beyond  the  common  tendency  accounted  for  by  link 
- is  significant.  Specifically,  in  the  forum  community  the  odds  of  setting  up  a response  tie  (i  —>  j)  increases  (by 
1,280)  if  actor  i is  N18,  the  main  responder  in  this  community.  In  contrast,  the  personal  class  of  the  tutor's  responses 
in  the  team  community,  respi,  is  statistically  insignificant,  respi  neighborhoods  are  therefore  not  significant  in  the 
explanation  of  the  behavior  of  the  team  community.  This  simply  means  that  the  tutor  of  the  team  community,  PI, 
showed  no  tendency  to  respond.  Hypothesis  H10  - the  tutor’s  residual  responsiveness  is  significant  - is  accepted  for 
the  forum  community  but  rejected  for  the  team  community. 

This  difference  is  attributed  to  the  difference  in  the  role-assignment  design  of  the  two  communities,  which 
leads  to  different  responsibility  mechanisms.  The  tutor  of  the  forum  community  was  assigned  the  job  of  responder. 
The  tutor  of  the  team  community  was  - deliberately  - not  assigned  that  role.  This  results  in  a difference  in  their 
responsibility  mechanisms  which  leads  to  the  difference  in  their  tendency  to  create  the  personal  class  of  virtual 
neighborhoods.  A similar  observation,  mentioned  above,  is  that  none  of  the  students  in  either  community  showed  a 
significant  personal  residual  tendency  to  respond,  which  supports  hypothesis  H8.  This  again  is  attributed  to  the  fact 
that  students  in  both  communities  were  not  controlled  by  responsibility  mechanisms  because  they  were  not  assigned 
any  particular  role.  Similarly,  in  both  communities  every  actor  could  trigger  others  by  posting  a question.  No  student 
was  pre-assigned  the  role  of  trigger.  This  is  reflected  in  the  insignificance  of  the  triggi  class  of  neighborhoods 
(consisting  of  a single  response  tie  towards  actor  i),  in  agreement  with  hypothesis  H9. 

We  see  that  the  tutors  in  both  communities  had  no  significant  tendency  to  trigger  others,  contrary  to 
assumption  HI  1.  Checking  the  designs  of  the  two  communities,  we  see  that  the  tutors'  behavior  was  not  controlled 
by  responsibility  mechanisms,  but  by  other  factors.  In  the  forum  community,  the  tutor  served  only  as  a helper  or 
responder;  no  initiation  of  discussion  was  designed;  accordingly,  no  triggering  role  was  assigned  to  the  tutor.  In  the 
team  community,  discussion  was  initiated  by  the  tutor,  but  the  design  of  the  collaborative  work  dictated  that  the 
tutor  should  step  aside.  The  tutor  was  therefore  not  responsible  for  triggering  others. 

Incorporating  the  out-stars  class  increases  the  goodness  of  fit  of  the  Markov  model  for  the  team  community 
by  5%  but  has  no  significance  for  the  forum  community.  This  means  that  in  the  team  community  the  likelihood  of 
forging  a response  tie  from  any  actor  i to  an  actor  j is  enhanced  (by  1.2)  if  the  tie  completes  an  out-star.  No  such 
tendency  is  observed  in  the  forum  community. 
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The  tendency  to  create  out-stars,  that  is,  to  forge  more  than  one  response  tie  can  be  explained  by  the 
contagion  theory  (hypothesis  H5)  and  the  theory  of  collective  action  (hypothesis  H2).  The  theory  of  contagion 
predicts  tendencies  toward  both  in- stars  and  mixed- stars,  but  these  predictions  were  not  supported  by  the  data  of 
either  community.  Thus,  hypothesis  H5  was  rejected  for  both  communities.  In  general,  contagion  by  exposure,  as 
found  in  friendship  relations,  is  a time  consuming  process,  which  presumably  could  not  be  developed  during  the 
short  lifetime  of  the  communities  (one  semester). 

The  hypothesis  concerning  the  theory  of  collective  action,  H2,  was  accepted  for  the  team  community  but 
rejected  for  the  forum  community.  This  theory  assumes  the  development  of  peer  pressure,  provided  that  the 
community  parameters  of  density  and  centrality  are  above  threshold  values.  This  condition  is  fulfilled  for  the  team 
community,  but  not  for  the  forum  community,  as  can  be  seen  in  Figures  2 and  3.  In  general,  developing  peer 
pressure  is  not  trivial,  as  it  has  to  overcome  the  basic  tendency  for  lurking.  In  the  team  community,  appropriate 
initial  conditions  - commitments,  interdependence,  and  in  particular  promotive  interactions  - were  set  up,  and  peer 
pressure  was  maintained  by  the  tight  schedule  of  common  sub-goals  imposed  on  the  community.  None  of  these 
features  were  designed  into  the  forum  community,  hence  the  density  and  the  number  of  central  actors  did  not  reach 
the  thresholds  required  for  peer  pressure  to  work.  In  the  absence  of  peer  pressure,  no  drive  for  collective  action 
arouse,  which  is  the  reason  for  the  non- significance  of  the  out-stars  class  of  virtual  neighborhoods  in  the  forum 
community.  The  differences  between  the  two  communities  in  the  tendencies  for  out-stars  is  explained  quite  well  by 
the  theory  of  collective  action. 

The  mutuality  class  of  virtual  neighborhoods  accounts  for  4%  of  the  goodness  of  fit  of  the  Markov  model 
for  the  forum  community.  It  has  no  significance  for  the  team  community.  This  means  that  in  the  forum  community 
the  likelihood  of  setting  up  a response  tie  from  any  actor  i to  any  actor  j is  enhanced  (by  5,000)  if  that  tie  closes  a 
mutual  tie.  (As  stated  elsewhere  in  this  paper,  the  actual  number  is  not  precise).  This  is  relative  to  the  likelihood  of 
setting  up  a tie  which  is  not  part  of  a mutual  tie.  No  such  tendency  for  mutuality  neighborhoods  exists  in  the  team 
community.  Thus,  hypothesis  H3  is  accepted  for  the  forum  community  but  rejected  for  the  team  community. 

Hypothesis  H3  predicts  a tendency  for  mutuality  virtual  neighborhoods  on  the  basis  of  the  exchange 
mechanism  postulated  by  the  theories  of  exchange  and  resource  dependency.  Actors  select  their  partners  for 
response  according  to  their  particular  resource-promising  state.  In  the  forum  community  the  actors  prefer  to  forge 
response  ties  (if  at  all)  with  partner(s)  who  usually  respond  to  them  - which  in  this  community  is  the  tutor.  The  tutor 
is  an  a priori  resource-promising  actor  as  result  of  her  pre-assigned  role.  This  kind  of  exchange  calculus  is  not 
developed  in  the  team  community  because  actors  in  that  community  cannot  identify  a priori  resource-promising 
actors.  Instead,  actors  in  the  team  community  chose  another  response  policy,  governed  by  the  cognition  balance 
mechanism,  of  responding  via  transitive  triads,  as  we  saw  above. 

The  in- stars  class  of  neighborhoods  accounts  for  3%  of  the  goodness  of  fit  of  the  Markov  model  to  the 
forum  community  but  has  no  significance  in  the  team  community.  From  Table  5 we  see  that  in  the  forum 
community  in- stars  is  negative.  In  the  forum  community,  the  likelihood  of  setting  up  a response  tie  from  i to  j 
decreases  if  this  tie  complements  an  in- star  neighborhood,  that  is,  if  some  other  actor  already  has  a response  tie  with 
j.  Contagion  theory  and  the  theory  of  uncertainty  reduction  both  predict  a positive  tendency  for  in-stars  virtual 
neighborhoods.  This  prediction  is  not  fulfilled.  Hypotheses  H5  and  H7  are  rejected  for  both  communities.  As 
mentioned  above,  the  fact  that  a contagion  process  did  not  develop  can  probably  be  attributed  to  the  short  lifetime  of 
the  communities  (one  semester).  In  addition,  it  seems  that  there  was  no  need  in  either  community  to  reduce 
uncertainties  by  attracting  responses  from  several  sources:  in  the  forum  community,  the  tutor  was  assigned  this  role; 
in  the  team  community,  the  rules  of  the  game  were  clearly  explained  in  the  document  detailing  the  design  of  the 
forum. 

We  have  yet  to  understand  the  negative  tendency  toward  in- stars  virtual  neighborhoods  in  the  forum 
community.  This  negative  tendency  means  that  participants  deliberately  avoid  responding  again  to  the  same  actor. 
This  phenomenon  is  explained  by  the  theory  of  social  capital:  responding  again  to  an  actor  is  a waste  of  energy;  it 
decreases  the  structural  autonomy  of  the  responder. 

Neither  community  shows  a tendency  for  mixed- stars  or  cyclicity  classes  of  virtual  neighborhoods,  mixed- 
stars  is  predicted  by  the  contagion  theory,  hypothesis  H5;  the  tendency  for  cyclicity  is  predicted  by  the  theory  of 
generalized  exchange,  hypothesis  H4.  Both  hypotheses  were  rejected  for  both  communities.  As  mentioned  above,  it 
is  plausible  that  the  contagion  mechanism  could  not  develop  during  the  short  lifetime  of  the  communities.  The 
theory  of  generalized  exchange  relies  on  knowledge  transfer  through  intermediaries,  who  seem  to  be  unnecessary  in 
online  broadcast  communities. 

Our  findings  are  summarized  in  Table  5. 
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Predicted  Hypotheses  and  Tendencies 

Results 

HI:  link  < 0 
Few  single  tie  links 

Supported  for  both  communities;  feature  of 
every  broadcast  community  independent  of 
design 

H2:  If  large  density,  centralization,  and  size,  then  out- 
stars  > 0 

Respond  to  several  others 

Supported  in  team,  but  not  in  forum;  difference 
in  meeting  threshold  conditions  due  to  built- 
in/lack  of  promotive  interactions 

H3:  mutuality  >0 

Tendency  to  reciprocate  to  resource  promising  partners 

Supported  in  forum  but  not  in  team;  difference 
in  existence/non-existence  of  a priori  resource- 
promising  actors  due  to  pre-assigned  roles 

H4:  cyclicity  > 0 

Tendency  to  respond  cyclically  to  resource-promising 
partner 

Not  supported  in  either  community,  probably 
because  there  was  no  need  for  information 
exchange  via  mediators 

H5:  out- stars  > 0;  in- stars  > 0;  mixed- stars  > 0; 
transitivity  > 0 

Respond  to  same  as  other  equivalent  actors 

Not  supported  in  either  community,  probably 
because  contagion  process  could  not  develop  in 
the  short  lifetime  of  the  communities 

H6:  transitivity  > 0 
Respond  via  several  paths 

Supported  in  team,  but  not  in  forum;  difference 
due  to  difference  in  consensus  reaching 
requirements  and  interdependence 

H7:  in-stars  > 0 

Attract  responses  from  several  others 

Not  supported  in  either  community; 
uncertainties  were  clarified  by  the  design  (in 
team)  and  by  the  tutor  (in  forum) 

H8:  {respi  = 0 1 i G students} 

H9:  { triggi  = 0 1 i G students} 

H10:  {respi  > 0 1 i = tutor} 

Hll:  {triggi  > 0 1 i = tutor} 

Residual  personal  tendencies  to  respond  or  trigger  only 
to  actors  with  pre-assigned  roles 

H8,  H9:  Supported  for  both  communities;  no 
pre-assigned  role  of  responders  to  students 
H10:  Supported  in  forum,  but  not  in  team; 
differences  due  to  differences  in  pre-assigned 
roles  of  the  tutor 

Hll:  not  supported  for  either  community;  no 
pre-assigned  role  of  triggers  to  students 

Table  5:  Summary  of  Results 

Discussion 

Our  analysis  shows  that  the  minimal-effort  hunt-for-social-capital  mechanism,  predicted  by  the  theory  of 
social  capital  & transaction  costs  controls  a large  part  of  the  behavior  of  both  communities:  a negative  tendency  to 
respond.  This  is  a feature  of  every  broadcast  community,  independent  of  design. 

Differences  in  the  goals,  interdependence,  and  the  promotive  interaction  features  of  the  designs  of  the  two 
communities  lead  to  the  development  of  different  mechanisms:  cognition  balance,  predicted  by  the  balance  theory, 
and  peer  pressure,  predicted  by  the  collective  action  theory  developed  in  the  team  community,  but  not  in  the  forum 
community.  An  exchange  mechanism  developed  in  the  forum  community,  but  not  in  the  team  community.  In 
addition,  the  unique  pre-assigned  role  of  the  tutor  in  the  forum  community  gave  rise  to  the  responsibility  mechanism 
in  that  community,  but  not  in  the  team  community.  The  differences  in  the  mechanisms  led  to  the  formation  of 
different  sets  of  virtual  neighborhoods,  which  show  up  macroscopically  as  differences  in  cohesion  and  in 
distribution  of  response  power.  These  differences  are  associated  with  the  differences  in  the  construction  of 
knowledge  in  the  two  communities  (Aviv  et  al.,  2003). 

It  should  be  noted  that  the  important  contagion  mechanism  did  not  develop  in  either  community.  This 
mechanism,  if  developed,  would  have  led  to  social  influence  and  imitation  in  attitudes,  knowledge,  and  behavior, 
which  would  have  developed  all  kinds  of  star  virtual  neighborhoods.  The  required  design  parameters  - promotive 
interaction  - were  in  place  in  the  team  community,  but  it  seems  that  the  lifetime  of  the  community  was  too  short  for 
the  development  of  this  mechanism. 

There  are  obvious  limitations  to  the  conclusions  drawn  here.  First,  we  have  considered  only  two 
communities.  In  order  to  capture  the  commonality,  as  well  as  the  differences  in  design,  neighborhoods,  and 
mechanisms  of  online  communities,  one  needs  to  consider  a larger  set  of  communities  of  different  sizes,  topics,  and, 
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in  particular,  with  different  designs.  Furthermore,  one  should  consider  a set  of  relations  embedded  in  these 
communities.  One  possibly  relevant  relation  between  actors  is  common  interest,  which  can  be  captured  by  common 
keywords  in  the  transcripts  and/or  common  sets  of  visited  web-pages. 

Another  limitation  lies  in  restricting  ourselves  to  Markov  neighborhoods.  Pattison  and  Robbins  (Pattison 
and  Robbins  2002)  emphasized  the  possible  importance  of  non-Markovian  neighborhoods  and  brought  initial 
evidence  for  the  empirical  value  of  models  that  incorporate  such  neighborhoods.  Thus,  the  dependence  structures 
can,  and  perhaps  should,  be  treated  as  a hierarchy  of  increasingly  complex  dependence  structures. 

It  seems  that  SNA  can  be  a useful  research  tool  for  revealing  community  architectures  and  mechanisms  of 
online  communities.  There  are  numerous  directions  for  future  research.  One  direction  is  “community-covariates 
interaction.”  Several  studies,  such  as  (Lipponen,  Rahikainen  et  al.  2001),  revealed  that  certain  participants  take  on 
the  roles  of  influencers  (who  trigger  responses)  or  of  celebrities  (who  attract  responses).  Others  are  isolated  - no-one 
responds  to  them  or  is  triggered  by  them.  The  question  is  whether  this  behavior  depends  on  individual  attributes  or 
whether  this  is  universal  and  found  across  communities.  Another  direction  is  “community  dynamics,”  an  inquiry 
into  the  time  development  of  community  structures.  When  do  cliques  develop?  Are  they  stable?  What  network 
structures  determine  this  behavior?  Yet  another  direction  is  “large  group  information  overload.”  It  is  well  known 
that  the  dynamics  of  large  groups  leads  to  boundary  effects  that  occur  when  the  group  and/or  the  thread  size 
increases  (Jones,  Ravid  et  al.  2002).  How  are  these  manifested  in  online  communities? 
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Discussion  Boards 


Dennis  Beck 

Discussion  boards  are  being  used  widely  in  a variety  of  educational  contexts.  It  makes  sense  then  to 
explore  how  the  composition  of  discussion  board  messages  influences  the  response.  This  study  explores  the 
characteristics  of  discussion  board  postings  that  seem  to  increase  the  replies  per  number  of  views. 

Discussion  Boards 

Much  research  has  been  done  concerning  the  effectiveness  of  discussion  boards  in  online  learning 
(Cavalier,  1992;  Collins,  1998;  Hoadley  & Linn,  2000).  Several  findings  were  of  note.  First,  collaborative 
discussion  adds  value  to  online  learning  occurring  in  a social  context  (Hoadley  & Linn,  2000).  Also,  students  spend 
more  time  online  learning  in  discussion  boards  than  they  do  learning  through  an  online  textbook  (Hoadley  & Linn, 
2000).  Use  of  discussion  boards  seem  to  promotes  interaction  and  increased  communication  among  students  and 
faculty  (Collins,  1998;  Coombs,  1992;  Zack,  1995).  They  also  seem  to  prolong  course  discussions  beyond  the 
synchronous  time  period  and  motivate  even  normally  silent  students  to  participate  in  discussions  (Schoenfield, 
1993).  Discussion  boards  provide  a place  for  students  to  discuss  their  internal  discourse  and  share  the  results  of 
their  generative  strategies  (Morrison  & Guenther,  2000).  They  also  foster  the  development  of  online  learning 
communities  (Egan  & Gibbs,  1997).  These  collaborative  learning  groups  are  useful  when  students  need  to  create 
new  models  or  correct  misconceptions  or  misunderstandings  (O’Malley  & Scanlon,  1990). 

Ferdig,  Roehler,  and  Pearson  (2002)  mention  intertextuality  and  level  of  engagement  as  two  qualitative 
measures  to  analyze  discussion  board  postings.  Intertextuality  is  the  degree  to  which  responses  mention  multiple 
texts,  experiences  and  examples.  A high  level  of  engagement  is  shown  when  a student  responds  to  a posting  by 
justifying  his  or  her  response  rather  than  simply  reiterating  another’s  position.  Their  research  found  “. . .a  high 
correlation  between  the  use  of  a discourse  forum  to  provide  critical  conversations  and  creation  of  a comparably 
complex  and  critical  written  product.”  (Ferdig,  Roehler  & Pearson,  2002,  p.  1 1).  One  of  the  implications  of  their 
research  is  that  discussion  boards  may  provide  learners  with  “. . .numerous  opportunities  for  posting,  reflection,  and 
the  internalization  of  key  ideas”  (Ferdig,  Roehler  & Pearson,  2002,  p.  12).  Based  on  this  finding,  I determined  to 
discover  what  characteristics  of  discussion  board  postings  would  enhance  these  numerous  opportunities  for  posting. 
However,  the  scope  of  Ferdig,  Roehler  & Pearson’s  article  did  not  include  whether  the  presence  of  intertextuality 
and  high  level  of  engagement  in  a discussion  board  posting  elicited  more  responses  per  number  of  views  than  those 
postings  that  did  not  include  intertextuality  and  high  level  of  engagement.  My  research  intended  to  discover 
whether  a high  incidence  of  intertextuality  and  high  level  of  engagement  may  elicit  more  responses  per  number  of 
views,  as  well  as  uncover  any  other  characteristics  of  discussion  board  postings  that  may  elicit  more  responses  per 
number  of  views. 


Research  questions 

In  this  analysis  I investigated  which  characteristics  of  online  discussion  board  postings  may  produce  more 
responses  per  number  of  views.  I also  investigated  whether  teachers  who  use  these  characteristics  in  their  discussion 
board  postings  elicit  more  responses  per  number  of  views.  I hoped  to  find  out  what  kinds  of  messages  teachers  are 
posting,  as  well  as  how  they  were  being  answered. 

More  research  on  discussion  boards  is  needed.  Specifically,  studies  into  what  types  of  postings  may  elicit 
certain  kinds  of  responses  would  be  helpful  in  promoting  discussion  boards  that  develop  students’  ability  to  think 
critically.  It  is  with  this  consideration  that  I undertook  this  study. 

This  qualitative  study  focused  on  the  online  interactions  between  those  who  identify  themselves  as  ESL  and 
EFL  instructors  on  the  website:  www.eslcafe.com  (Dave’s  ESL  Cafe).  Specifically,  I looked  at  instructors  whose 
expert  use  of  discussion  boards  promotes  more  replies  per  number  of  views  than  other  users.  An  expert  poster  was 
identified  when  an  instructor  was  seen  to  have  received  more  replies  per  number  of  views  than  the  average  of  other 
posters. 


Background  and  process 

Discussion  boards  are  places  in  the  web  where  people  can  go  to  and  interact,  ask  or  answer  questions,  and 
share  ideas  on  specific  topics.  Several  discussion  boards  are  housed  in  Dave  \s  ESL  Cafe.  The  discussion  boards  in 
Dave's  ESL  Cafe  are  common  to  discussion  boards  in  general.  They  provide  an  asynchronous  format  where  site 
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members  may  post  a topic  and  responses.  The  responses  within  a consistent  topic  form  a "thread."  A “response”  is 
recorded  when  an  instructor  clicks  the  reply  button,  types  their  message,  and  then  clicks  on  the  submit  button.  A 
“view”  is  recorded  when  another  instructor  views  the  posting,  regardless  of  whether  or  not  they  reply  to  it. 

This  research  studied  the  messages  in  the  on-line  public  discussion  boards  called  Applied  Linguistics 
(http : //www . eslcafe . com/forums/teacher/vie  wforum.php  ?f=  3 ) and  Activities  and  Games 

(http://www.eslcafe.com/forums/teacher/viewforum.php?f=  1 ).  Messages  were  collected,  characteristics  identified, 
and  comparisons  made,  using  the  constant  comparative  method  (Glaser  & Strauss,  1967).  Participant’s  names  were 
changed  so  that  anonymity  is  guaranteed.  The  Activities  and  Games  board  and  the  Applied  Linguistics  board  in  the 
ESL  Cafe  were  chosen  specifically  because  one  is  a specific  content  area  (applied  linguistics)  and  the  other  is  an 
activities  area  (Activities  and  Games).  Thus,  they  provided  a sampling  of  two  different  types  of  boards  (content  and 
activities)  within  the  ESL  asynchronous  discussion  boards. 

Research  Locale 

This  site  was  chosen  because  the  site  content  and  discussion  boards  seemed  representative  of  other 
academically  oriented  sites  on  the  web.  The  site  is  run  by  Dave  Sperling.  During  the  last  17  years,  he  has  been  a 
Professor  of  advanced  English  subjects  at  California  State  University,  Northridge.  He  is  recognized  as  an  expert  in 
ESL  methodology  on  a global  scale  as  he  participates  in  numerous  national  and  foreign  conferences.  ESL  Cafe 
focuses  in  assisting  worldwide  ESL  (English  as  Second  Language)  and  EFL  (English  as  Foreign  Language)  students 
and  teachers.  This  web  site  offers  links  to  all  types  of  educational  resources,  including  articles,  online  ESL 
publications  and  courses,  teacher  training,  grammar,  and  writing.  The  site  has  a section  devoted  to  bulletin  boards, 
an  asynchronous  format  where  site  members  may  post  a topic  and  responses. 

Data  was  gathered  by  first  identifying  several  instructors  whose  postings  generated  more  responses  per 
number  of  views  than  the  average  of  other  posters.  This  was  discovered  by  counting  the  threads,  number  of 
postings,  and  number  of  views  per  instructor  during  the  week  in  question.  After  averages  were  calculated,  the 
instructors  who  ranked  in  the  top  five  in  terms  of  rate  of  responses  per  number  of  views  were  selected  as  our  expert 
posters.  Non-expert  posters  were  identified  as  those  whose  rate  of  responses  per  number  of  views  was  below  the 
average  of  all  posters.  The  responses  of  the  expert  posters  were  collected  by  logging  on  to  the  site,  following  the 
threads  of  each  instructor,  and  copying  all  items  in  a thread.  Data  was  collected  from  both  the  Applied  Linguistics 
and  Activities  and  Games  forums  during  fall  2004  for  one  week.  This  yielded  approximately  30  threads  per 
instructor  with  anywhere  from  5 to  20  messages  in  each  thread.  Similar  data  was  also  collected  from  an  equal 
number  of  non-expert  or  average  instructors. 

The  data  were  analyzed  by  first  reading  the  threads  associated  with  each  selected  instructor.  It  was  through 
reading  these  threads  that  several  specific  characteristics  began  to  emerge.  Data  from  other  instructors  whose 
postings  did  not  elicit  the  same  rate  of  responses  were  then  collected,  read,  and  analyzed  through  the  application  of 
these  specific  characteristics.  During  the  week  in  question,  instructors  began  13  topics,  made  206  postings  and 
viewed  2852  postings  in  the  applied  linguistics  forum  (.0676  replies  per  view).  In  the  Activities  and  Games  forum, 
instructors  began  5 topics,  made  27  postings  and  viewed  2213  postings  (.0099  replies  per  view).  Our  expert  posters 
averaged  .1079  replies  per  view. 

Table  1 


Topics 

Postings 

Views 

Postings  per  view 

Applied 

Linguistics 

Forum 

13 

206 

2852 

.0676 

Activities  and 
Games  Forum 

5 

27 

2213 

.1079 

Dave’s  ESL  Cafe  is  designed  so  that  the  topics  or  threads  are  divided  into  categories,  which  can  include 
everything  from  activities  and  games  to  video  in  the  classroom.  In  each  topic,  a member  may  begin  a thread  by 
posting  a question  or  statement  he  or  she  would  like  answered  or  commented  on  in  the  forum  of  choice.  Many  of  the 
topics  or  threads  were  those  that  would  be  considered  relating  to  practical  or  academic  issues  related  to  ESL  and 
EFL  instruction.  One  thread  followed  throughout  the  week  concerned  the  discussion  of  the  quote,  "Pragmatic 
failure  has  more  potentially  devastating  consequences  than  linguistic  failure."  (Braden,  taken  from 
http://www.spanport.ucsb.edu/proiects/llcf/Eral/Reviews/Susanb/Matsudainterlanguage.htmk  ERAL,  E-reviews  in 
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applied  linguistics  and  language  learning  on  November  19,  2004).  On  this  thread,  teachers  discussed  whether  they 
agreed  or  disagreed  with  the  quote  and  why,  citing  other  references,  resources,  and  examples  when  appropriate.  This 
topic  was  used  to  generate  discussion  about  the  importance  of  pragmatics  and  give  support  to  instructors  struggling 
in  how  to  teach  it. 


Investigation 

Dave’s  ESL  Cafe  publishes  the  messages  in  a public  format  that  allows  anyone  who  finds  the  site  to  read 
them.  Thus,  the  data  were  collected  by  observing  and  following  threads;  as  a researcher,  I did  not  make  my  presence 
known,  nor  did  I interact  with  or  contact  site  members  in  any  way.  As  part  of  the  collection  procedure  to  protect  the 
confidentiality  of  the  members,  all  screen  names  and  dates  of  posting  were  immediately  removed  from  each 
message.  I chose  to  observe  rather  than  ask  for  permission  because  of  the  public  nature  of  the  site,  and  the  concern 
that  identifying  myself  as  a researcher  might  change  the  nature  of  the  interaction  online.  This  means  that  the 
members  of  the  website  were  and  are  ignorant  of  my  presence  as  a researcher.  This  study  is  limited  to  the  single  data 
source  of  the  posts  by  members.  While  these  posts  provide  a rich  data  source,  they  do  not  allow  for  any  interaction 
between  the  researcher  and  the  site  participants.  Therefore,  the  analyses  provided  do  not  allow  for  ethnographic 
analyses.  Further  research  in  this  area  must  make  use  of  other  observation  procedures  in  order  to  better  understand 
the  multifaceted  character  of  online  discussion  board  postings,  as  well  as  to  give  expression  to  the  individuals 
posting  the  messages  in  an  open  interface  with  the  investigator.  Although  the  possibility  exists  to  do  research  on  the 
Internet  in  publicly  accessible  areas,  depending  on  the  situation,  it  may  be  more  revealing  to  do  so  overtly.  Future 
observation  procedures  which  may  yield  interesting  results  are  anthropological  and  ethnographical  in  nature. 
Research  into  the  historical  background  of  certain  quotes,  references,  and  examples  which  yielded  more  responses 
per  number  of  views  may  produce  interesting  outcomes  in  terms  of  how  it  has  been  analyzed  and  commented  on 
previously,  and  how  those  historical  comments  are  similar  or  different  in  structure  to  present  day  commentary. 

Also,  immersion  into  the  culture  of  discussion  boards  may  yield  interesting  insights  into  the  thought  processes  that 
contribute  to  writing  an  expert  posting. 


Data  Analysis 

Why  did  bobthelinguist , Andrea  Jones , Jennagirl , punkrockartist , and  Sally  Smith’ s (names  have  been 
changed)  postings  elicit  more  responses  per  number  of  views  than  other  users  (from  this  point  forward  I shall  refer 
to  the  above  mentioned  users  as  our  expert  posters)?  Discussion  boards  can  promote  interaction  and  increased 
communication  (Collins,  1998;  Coombs,  1992;  Zack,  1995),  and  foster  analytical  and  evaluative  conversations 
(Ferdig,  Roehler  & Pearson,  2002)  while  providing  a place  for  students  to  discuss  their  internal  discourse  and  share 
the  results  of  their  generative  strategies  (Morrison  & Guenther,  2000).  As  a result,  it  makes  sense  to  explore  the 
question  of  why  our  expert  posters  elicited  more  responses  per  number  of  views  than  other  users.  In  other  words, 
what  common  characteristics  or  themes  might  cause  this  increase  in  responses,  and  are  these  characteristics 
consistent  with  current  research? 

1.  Number  of  Sentences  in  a posting 

As  I analyzed  the  data,  several  common  characteristics  were  identified  in  the  postings.  First,  our  expert  posters 
averaged  ten  sentences  in  their  postings.  The  average  length  of  postings  for  non-experts  was  six  sentences. 
Although  length  of  postings  was  not  uniform  in  the  non-expert  group,  they  were  uniform  among  our  expert  posters. 

2.  Content  specific  postings 

Second,  our  expert  posters  were  very  content  specific  in  their  postings.  For  example,  consider  the  following 
postings  by  one  of  our  expert  posters  (. Jennagirl ) and  a non-expert  poster  (Eddie): 

Jennagirl: 

“This  article  is  a clear  reiteration  of  what  we  have  all  heard  before.  How  about  trying  to  deal  with  some  of  the 
criticisms  instead?  For  example,  that  troublesome  one  on  the  other  thread  about  how  any  old  nonsense  creates  a 
polite  'distant'  sentence,  whether  it  be  a past  tense  form  or  not. 

Anyway,  is  this  article  surprising  to  anybody?  Does  anyone  really  think  that  tense  and  aspect  are  generated  by  a 
precise  location  on  a rigid  time  chart,  rather  than  by  perspective  and  temporally  linked  context?” 

Eddie: 

“The  mainstream  has  explained  Present  Perfect  with  since.  ©” 

Notice  how  content  specific  Jennagirl  is  in  her  postings.  Now  compare  this  with  the  post  by  a non-expert  group 
poster,  Eddie.  While  Jennagirl  gives  a rich  context  and  specific  content  for  her  question,  Eddie  simply  makes  a 
statement  without  context.  This  content  specific  posting  elicited  more  responses  per  number  of  views  than  Eddie ’s. 

3.  The  use  of  examples,  quotes,  and  references 
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Another  common  characteristic  in  the  expert’s  postings  was  the  use  of  plenty  of  examples,  quotes,  and  references. 
Note  the  following  examples  by  bobthelinguist  (edited  for  brevity)  and  the  non-expert  poster,  Frodo : 

Bobthelinguist : 

“’’Critical  period"  only  makes  total  sense  to  me  in  the  context  of  feral  "wolf  children"  and  the  like  (that  is,  if 
children  aren't  exposed  to  ANY  language  before  a certain  age,  they  seem  to  have  have  little  hope  of  ever  being  able 
to  learn  a language  and  make  much  sense  of  the  world). 

His  (Krashen’s)  ideas  about  "comprehensibility"  (of  input,  which  can  aid  "acquisition"  processes)  are  useful  to 
consider,  if  only  because  approaches  can  often  be  either  too  "mindless"  (repetitive,  and  not  addressing  meaningful 
contexts  very  much)  or  too  "mindful"  (explicitly  studying  grammar  and  pondering  finer  distinctions  of  meaning  than 
is  perhaps  necessary). 

Whether  he  really  has  a point  about  "acquisition"  versus  "learning"  absolutely  is  another  point  entirely,  however  (I 
recall  Jeremy  Harmer  providing  a good  discussion  of  this  in  the  Second  Edition  of  his  The  Practice  of  English 
Language  Teaching );  I think  the  important  thing  is  merely  that  we  come  to  possess  insight  or  knowledge,  and  both 
formal  and  informal  encounters  can  lead  to  pretty  much  the  same  thing  somehow  ending  up  in  our  heads  (especially 
if  the  context  as  presented  in  a classroom  were  an  informal  context  itself)! 

In  my  own  experience,  I just  know  that  I needed  to  have  some  idea  of  what  was  possible  form- wise  (from  formal, 
book  learning)  to  make  total  sense  of  and  really  appreciate  whatever  informal  language  spontaneously  came  my 
way,  so  I think  Krashen  is  being  a bit  silly  when  he  says  that  leaming/studying  is  unnecessary  or  almost  a bad  thing. 
Even  those  people  I've  met  who  professed  to  have  picked  up  a language  without  any  formal  study  turned  out  to  have 
often  asked  questions  about  meaning  of  their  conveniently  bilingual  wives  (their  "walking,  talking  dictionaries"!). 
Frodo: 

“Does  anybody  can  help  me?  I'd  need  some  Halloween  crafts  to  do  with  my  little  students  (from  5 to  8 
years  old),  so  they  should  be  easy  to  do  and  cheap.  Many  thanks  in  advance” 

Notice  that  bobthelinguist  uses  several  personal  examples  (in  his  comments  about  learning  on  the  street,  in 
comments  on  the  need  for  studying,  etc.),  and  references  other  sources  (Jeremy  Harmer,  Krashen),  while  Frodo  only 
asks  a question.  The  examples  and  references  used  by  bobthelinguist  were  referred  to  in  several  other  user 
responses  and  seemed  to  promote  more  responses  per  number  of  views  than  postings  that  did  not  use  examples  and 
references. 

4.  Quoting  parts  of  previous  postings 

When  our  expert  posters  quoted  parts  of  previous  posts  before  answering,  the  result  seemed  to  be  more 
responses.  For  example,  Andrea  Jones  said: 

“Quote:  How  are  we  with:  It  was  a very  fine  day. 

Yes,  I'm  fine  with  that. 

I think  that : 

1)  "fine"  when  talking  about  threads  or  material  means  either  gosamer-like  or  high  quality. 

2)  When  talking  about  other  people  or  the  weather  it  means  high  quality/great/nice. 

Both  these  uses  are  gradable. 

This  was  not  a common  practice  among  non-expert  posters.  Quoting  part  or  all  of  a previous  posting  seems  to 
provide  a link  from  the  current  posting  to  the  one  being  quoted.  Suler  says  that  this  technique  may  lead  to 
“. . .interesting,  interweaving,  multi-layered  dialogue.”  (2004,  p.  399).  The  link  is  clearer  to  the  reader  and  is  more 
likely  to  be  commented  on  in  future  postings,  resulting  in  more  responses  per  number  of  views. 

5.  Contributing  to  an  ongoing  story 

Another  common  characteristic  in  the  expert’s  postings  is  that  their  postings  seemed  to  contribute  to  an 
ongoing  story.  Here’s  one  example: 

“I  attended  an  interview  recently  to  teach  at  a small  school  where  the  majority  of  my  students  would  be 
kids  between  the  ages  of  3 and  junior  high  school  age. 

After  some  basic  fact-finding  Q&As,  the  manager  of  the  school  and  her  sole  foreign  teacher  handed  me  a 
pile  of  food  and  drink  flashcards  (ice  cream  cone,  slice  of  pizza,  glass  of  OJ  and  the  like)  and  asked  me  to 
demonstrate  how  I would  "teach  the  difference  between  like  and  want". 

Now  call  me  a slacker  and  inexperienced  despite  my  years  in  the  "profession",  but  I must  admit  that  I was  a 
bit  stumped  (then  again,  what  can  anyone  expect  given  zero  preparation  time?  It's  not  like  I don't  usually 
prepare  for  my  lessons!). 

My  initial  reaction  as  I heaved  myself  up  to  do  "something"  (anything!)  was  that  want  is  the  far  less 
appropriate  or  polite  (at  least  in  its  probable  relation  to  foodstuffs!)  than  like  (and  after  a bit  more  thought,  I 
think  our  wants  are  much  "deeper"  and  not  as  easily  or  openly  expressed  as  our  likes).  Anyway,  in  this 
context,  I reckon  like  was  by  far  the  better  thing  to  teach,  why  introduce  want  at  all? ! 
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I am  sure  we  are  all  familiar  with  the  "host  offering  refreshments  to  visitors"  script: 

Host:  Do  you  want/Would  you  like  anything  to  eat/drink?  (I  have  OJ,  coke. ..or  do  you  want  a beer?  etc). 
Visitor:  Can  I have...  (?I'd  like...;  *1  want...) 

So,  ultimately,  I thought  the  demonstration  question  was  more  revealing  of  how  my  potential  boss  would 
teach  it  than  how  I would  (or  wouldn't)  go  about  things.  I could  envisage  the  kind  of  lesson  they  wanted: 
me  maniacally  waving  flashcards  and  shouting  "I'm  hungry!  I like  bananas!  I want  a/the  (?)  banana!  NOW, 
dammit! ! !".  I won't  presume  to  bore  you  with  how  exactly  I winged  my  way  through  it  before  crash-landing 
(besides,  I can't  really  remember  much).©” 

The  expert  poster  punkrockartist  provided  a personal  story  to  set  up  his  question.  Story  seems  to  possess 
an  inherent  ability  to  make  a case  without  provoking  intellectual  opposition  (Friedlander,  1992;  Slater,  1990;  Slater 
& Rouner,  1996,  1997).  Story  may  also  provoke  less  inquiry  through  the  temporary  deferral  of  skepticism 
(Graesser,  1981;  Slater,  1990)  and  connection  with  the  central  character’s  outlook  (De  Vega,  Diaz,  & Leon,  1997; 
Slater  & Rouner,  1997).  People  regularly  utilize  story  to  show  their  positions  and  to  persuade  others  of  the 
soundness  of  their  position  (Berger,  1997;  Fisher,  1984,  1985).  There  are  multiple  ethnically  oriented  groups  that 
utilize  story  to  enculturate  members  into  their  group.  This  stems  from  an  educational  viewpoint  that  story  will  lead 
to  greater  retention  of  information  than  isolated  statements  (Bruchac,  1996).  Consistent  with  this  literature,  the 
result  in  this  case  was  that  our  expert  poster,  punkrockartist,  received  more  replies  per  number  of  views  than  other 
users  who  merely  posted  a question. 

6.  Contributing  to  own  thread  after  initial  posting 

The  last  common  characteristic  I examined  was  that  the  expert  posters  seemed  to  contribute  to  their  own 
thread  after  their  initial  post  much  more  than  other  posters.  In  one  particular  thread,  after  her  initial  post,  Andrea 
Jones  responds  three  other  times  in  a thread  containing  16  posts.  Compare  this  to  the  non-expert  poster  Frodo , 
whose  only  contribution  to  the  thread  is  his  initial  posting.  Andrea  Jones’  practice  displays  a high  level  of 
interactivity  and  high  level  of  engagement.  She  is  not  merely  restating  what  others  say,  but  she  is  clarifying  and 
justifying  her  posting.  Other  users  replying  in  the  thread  seem  to  appreciate  her  attention  to  the  topic,  and  are 
spurred  on  to  post  more  replies. 

Discussion  and  Conclusions 

The  common  characteristics  identified  in  the  postings  of  those  users  who  regularly  elicited  the  most 
responses  were  a)  User  contributes  to  an  ongoing  discussion/story,  b)  User  contributes  to  own  thread  after  initial 
post,  c)  User’s  posting  was  very  content  specific,  d)  User  quotes  parts  of  previous  posting  before  responding,  e)  User 
averaged  ten  sentences  or  more  in  postings,  and  f)  User  regularly  used  examples,  quotes,  and  references.  From  this 
study  on  these  ESL  discussion  boards,  it  can  be  extrapolated  that  these  are  common  characteristics  that  should  be 
present  in  ESL/EFL  discussion  board  postings  on  the  whole.  However,  these  characteristics  of  postings  that 
promote  more  responses  per  number  of  views  cannot  be  applied  to  other  discussion  boards  without  further  cross- 
topical  studies  to  confirm  these  results.  Future  studies  may  provide  this  needed  confirmation,  as  well  as  answer  the 
question  of  why  each  characteristic  elicited  more  responses. 

As  stated  in  the  introduction,  it  was  my  belief  that  a high  incidence  of  intertextuality  and  high  level  of 
engagement  may  promote  more  responses  to  a discussion  board  posting.  Our  study  showed  that  expert  posters 
regularly  used  examples,  quotes,  and  references.  This  would  seem  to  indicate  that  intertextuality  may  promote  more 
responses  to  a discussion  board  posting.  Also,  our  study  showed  that  expert  posters  regularly  contribute  to  their  own 
thread  after  their  initial  post,  providing  both  clarification  and  justification  for  their  initial  posting.  This  seems  to 
indicate  that  a high  level  of  engagement  promotes  more  responses  to  a discussion  board  posting. 

One  implication  of  these  findings  is  that  the  characteristics  found  are  connected.  As  noted  above, 
contributing  to  your  own  thread  after  the  initial  post  seems  to  be  evidence  of  a high  level  of  engagement.  The 
content  specific  posting  characteristic  may  also  contribute  to  a high  level  of  engagement,  as  the  presence  of  specific 
content  was  often  included  statements  of  justification  and  clarification.  Further,  quoting  parts  of  previous  postings 
before  responding  seemed  to  provide  an  intellectual  springboard  to  launch  other  opinions  or  arguments.  This 
characteristic  seems  related  to  both  intertextuality  and  a high  level  of  engagement.  The  average  of  ten  sentences  or 
more  in  postings  is  most  likely  the  result  of  instructors  who  have  reflected  deeply  on  a subject  and  thus  have  more  to 
contribute  than  most  users.  Contributing  to  an  ongoing  story  is  connected  to  intertextuality  in  that  the  presence  of 
multiple  texts,  experiences,  and  examples  are  often  present  in  story. 

Although  these  characteristics  need  to  be  confirmed  by  future  cross-topical  studies,  it  may  be  fruitful  for 
instructors  to  use  these  characteristics  in  the  creation  of  a discussion  board  posting  rubric.  Thus  a posting  that 
followed  this  rubric  may  help  promote  critical  thinking  and  metacognition  in  learners  (Ferdig,  Roehler  & Pearson, 
2002). 
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Another  difference  in  the  study  noted  was  the  significant  increase  in  responses  per  number  of  views  in  the 
Applied  Linguistics  board  compared  to  the  Activities  and  Games  board.  This  may  indicate  that  more  responses  are 
generated  for  a content  specific  discussion  board  (Applied  Linguistics)  than  one  that  is  cross-topical  (activities  and 
games).  However,  this  comparison  did  not  hold  true  when  comparing  the  responses  per  number  of  views  to  other 
discussion  boards  in  the  ESL  Cafe.  Another  possible  explanation  may  be  that  because  Applied  Linguistics  is  central 
to  ESL/EFL  instruction,  it  garnered  the  highest  responses  per  number  of  views.  Determining  whether  this  is  the  case 
would  be  a good  question  to  investigate  in  a future  study. 
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A Tool  for  Supporting  Urban  Teachers  in  Training 
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Introduction 

The  authors’  aim  is  to  help  novice  teachers  develop  as  purposeful  instructional  designers  and  reflective 
practitioners  - professionals  who  can  work  with  the  complexities  of  teaching  and  classroom  contexts  in  an  age  of 
rapidly  advancing  digital  technology  and  cultural  diversity.  This  paper  will  briefly  describe  the  context,  rationale, 
and  a conceptual  model  for  an  electronic  tool  that  will  help  address  this  goal. 

Context 

The  Crossroads  project  is  sponsored  by  the  Preparing  Tomorrow’s  Teachers  for  Using  Technology  (PT3) 
program  of  the  Department  of  Education.  Based  on  one  of  the  grant’s  objectives,  the  presenters  were  charged  with 
designing  and  ultimately  developing  an  online  support  environment  for  novice  teachers’  technology  integration  in 
urban  schools.  The  system,  TEPSS  (Teachers  Electronic  Support  System),  consists  of  four  fully  integrated 
components:  a)  An  instructional  design/lesson  planning  wizard  will  guide  the  users  through  designing  an  activity 
based  on  the  CITIS  ID  model  for  Culturally  Infused  Technology  Integration;  b)  a Video  Reflection  tool  will  guide 
students  through  editing  and  reflecting  on  video  footage  of  themselves  implementing  their  activities;  c) 
communications  tools  (Email,  BLOG,  Chatroom);  and  d)  information/resources  (systems  help,  instructional 
resources,  best  practices,  job  aids)  will  be  available  at  any  point  within  the  system. 

Rationale 

Novice  teachers  experience  a variety  of  challenges  not  limited  to  classroom  management,  subject  matter 
expertise,  teacher-parent  communications,  organization,  locating  resources,  and  developing  rapport  with  students. 
Preparing  to  teach  in  a high  need,  low  socio-economic  status  (SES)  school  can  significantly  increase  the  burden 
placed  upon  novice  teachers  in  urban  environments.  Sachs  (2004)  identified  five  attributes  of  effective  urban 
teachers  that  have  been  consistently  recognized  by  researchers:  a)  sociocultural  awareness  b)  contextual 
interpersonal  skills,  c)  self-understanding,  d)  risk-taking,  and  e)  perceived  efficacy.  According  to  a literature  review 
by  Hogan,  Rabinowitz  and  Craven  (2003),  there  are  several  important  differences  between  expert  and  novice 
teachers  in  the  area  of  pedagogy.  Unlike  expert  teachers,  novice  teachers  fail  to  visualize  planning  as  a scaffold  of 
events.  Instead,  they  perceive  lesson  planning  as  an  individual,  daily  episode  unconnected  to  the  curriculum  as  a 
whole.  Experienced  teachers  require  more  details  when  planning  a lesson  (such  as  availability  of  equipment,  student 
ability  and  prior  knowledge).  In  contrast,  novices  generally  disregard  these  details  and  proceed  to  develop  the 
lesson.  Experts  tend  to  use  multiple  assessment  strategies  throughout  a lesson  in  order  to  understand  students’ 
schema  before  introducing  new  material.  Novices  tend  to  teach  in  ways  that  disregard  the  importance  of  the 
connection  between  prior  and  new  knowledge.  While  experts  regard  the  classroom  as  a make-up  of  individuals  with 
personal  prior  knowledge  and  individual  differences,  novices  attach  a group  identity  to  the  class  and  base  their 
instructional  strategies  on  a general  level  of  ability,  knowledge,  interest  and  cultural  identity. 

Increasingly,  beginning  teachers  are  encountering  schools  with  students  diverse  in  culture,  ethnicity,  race, 
religion,  and  language.  Effective  teachers  who  meet  the  needs  of  these  diverse  learners  are  skilled  in  culturally 
responsive  pedagogy  (Gay,  2002;  Ladson-Billings,  1995).  According  to  Gay  and  Kirkland  (2003),  culturally 
responsive  teaching  includes  “using  the  cultures,  experiences,  and  perspectives  of  African,  Native,  Latino,  and  Asian 
American  students  as  filters  through  which  to  teach  them  academic  knowledge  and  skills”  (p.  181).  This  approach 
validates  students’  cultures  by  teaching  racially  diverse  students  to  have  a positive  cultural  identification  of 
themselves  and  others.  In  order  for  preservice  teachers  to  create  meaningful  instruction  that  is  culturally  relevant, 
they  need  to  develop  deeper  self-knowledge  about  how  their  own  backgrounds  and  beliefs  might  shape  their 
teaching  and  their  students’  self-  perceptions  (Howard,  2003). 

Reflection  is  an  important  part  of  professional  practice  and  professional  growth  (Dewey,  1933;  Schon, 
1983;  1987).  It  is  especially  important  in  the  development  of  culturally  relevant  pedagogy  (Howard,  2003).  Schon 
(1983)  suggested  that  there  are  two  types  of  reflection:  reflection-on-action  (looking  back  on  an  incident)  and 
reflection-in-action  (considering  the  incident  as  it  unfolds).  According  to  Schon,  reflection-in- action  depends  on  the 
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intuitive  performances  that  lead  to  surprises.  These  “surprises”  may  be  something  pleasing  and  desired  or  confusing 
and  unwanted.  Reflection-in-action  is  bounded  by  the  “action-present,”  the  amount  of  time  in  which  an  action  can 
impact  the  situation.  Killion  and  Todnem  (1991)  extended  the  ideas  of  Schon  to  include  reflection-for-action,  the 
desired  outcome  to  guide  future  action.  Thus  the  reflection  process  simultaneously  includes  past,  present  and  future 
timeframes. 

The  key  to  effective  reflective  practice  lies  in  helping  pre- service  teachers  look  beyond  the  “technical”  and 
contextual  aspects  of  teaching  to  questioning  their  knowledge  and  assumptions  (Van  Manen,  1977;  Gay  & Kirland, 
2003).  Ideally  the  level  of  reflection  parallels  the  growth  of  the  practitioner  from  novice  to  expert  (Van  Manen, 
1977).  However,  Ferry  and  Ross-Gordon’s  (1998)  research  on  links  between  experience  and  reflection  indicates  that 
experience  alone  may  not  promote  more  sophisticated  levels  of  reflection.  It  may  have  more  to  do  with  the  way  in 
which  the  educator  uses  the  experience  to  reflect  upon  specifics  of  a problematic  situation  that  fosters  development 
into  an  expert  as  apposed  to  an  experienced  non-expert. 

Although  teacher  education  programs  can  not  graduate  experts,  they  can  assist  pre-service  teachers  in 
developing  their  pedagogical  expertise  more  quickly  and  increasing  their  level  of  reflection.  As  emphasized  by 
Sparks-Langer  and  Colton  (1991),  “teachers  need  the  opportunities  to  construct  their  own  narrative,  context-based 
meaning  from  information  provided  by  research,  theoretical  frameworks,  or  outside  experts”  (p.43).  We  would  add 
that  these  narratives  can  and  should  be  connected  somehow  to  their  experiences  in  schools.  Instructional  design 
models  are  useful  tools  in  helping  novice  instructional  designers  to  think  more  like  an  expert.  They  provide  explicit 
directions,  steps  and  scaffolding  for  the  beginning  designer.  A design  model  which  addresses  the  pedagogical  gaps 
typical  of  novice  teachers,  the  complexity  of  technology  integration  and  the  tenets  of  reflective  practice  could  assist 
pre- service  teachers  with  lesson  design  in  urban  environments. 

Proposed  Model:  CITIS 

Culturally  Infused  Technology  Integration  Support  (CITIS)  is  a two-part  model  (see  figure  1).  One 
component  is  a scaffold  for  lesson/activity  design;  the  other  represents  a metacognitive  process  to  promote 
reflection.  The  authors’  aim  is  to  help  novice  teachers  develop  as  purposeful  instructional  designers  and  reflective 
practitioners  - professionals  who  can  work  with  the  complexities  of  teaching  and  classroom  contexts  in  an  age  of 
rapidly  advancing  digital  technology  and  cultural  diversity.  CITIS  blends  a learner-centered  philosophy  with  the 
proper  amount  of  guidance  needed  to  help  novice  teachers  become  more  comfortable  planning  activities  for  their 
classrooms.  CITIS  also  infuses  the  cultural  and  socioeconomic  awareness  necessary  for  working  in  urban 
environments.  This  is  not  a prescriptive  model,  rather  one  that  can  guide  novice  teachers  and  teacher  educators  in 
their  investigation  of  successful  teaching  practice. 

Part  1 contains  nine  elements  of  lesson  design  performed  in  five  stages.  In  stage  1,  the  preservice  teachers 
identify  the  curriculum  and  its  corresponding  standards  as  well  as  the  cultural  context  and  learner  characteristics. 
They  identify  the  students’  knowledge,  skills,  and  attitudes  related  to  the  curriculum  as  well  as  preferred  learning 
styles.  Curriculum  and  standards  information  may  be  obtained  from  local  and  state  curriculum  guides,  national 
educational  organizations,  textbooks,  school  faculty  and  other  resources.  After  reviewing  the  curriculum,  the 
beginning  teachers  consider  how  they  will  tailor  the  subject  matter  to  better  reflect  students’  ethnic  and  cultural 
diversity.  A review  of  the  materials  may  reveal  gender  or  cultural  bias  or  absence  of  a particular  culture’s 
representation  such  as  those  of  Indigenous  or  Latino  authors.  Supplemental  materials  may  need  to  be  added.  During 
this  time,  the  preservice  teachers  may  use  surveys,  pre-tests,  or  class  discussions  in  order  to  identify  students’  prior 
knowledge  and  skills  as  they  relate  to  short  stories,  such  as  story  elements  and  literary  devices,  as  well  as  interest 
inventories  to  measure  attitudes  such  as  those  towards  the  genre,  the  potential  themes  to  be  discussed,  reading  in 
general,  and  preferred  learning  styles. 

Once  the  initial  analysis  is  complete,  the  preservice  teachers  move  to  stage  2:  identify  the  long-term  goals. 
The  long-term  goals  reflect  both  the  analysis  of  the  learner  as  well  as  an  analysis  of  the  curriculum.  Repeating  the 
goals  for  the  unit  of  curriculum  each  time  the  daily  lesson  plans  are  written  is  designed  to  help  novice  teachers  avoid 
the  pitfall  of  dividing  lessons  into  discrete  pieces  of  instruction  (Hogan,  Rabinowitz  & Craven,  2003). 

At  stage  3 the  preservice  teachers  write  the  objectives  and  plan  the  assessments,  design  the  activities  and 
select  the  technology  and  materials  needed  for  the  lesson.  These  elements  are  recursive.  The  teachers  may  modify 
each  as  they  works  through  this  stage  of  the  design  process.  Teachers  often  list  performance  standards  as  their 
objectives;  thus,  CITIS  prompts  the  novice  teacher  to  rewrite  these  standards  into  measurable  outcomes  that  reflect 
their  learners’  characteristics  and  align  with  the  broad  curriculum  goals.  At  the  same  time,  the  preservice  teachers 
should  also  determine  what  evidence  they  will  use  to  determine  if  the  students  have  achieved  the  desired 
understanding.  Teachers  tend  to  think  of  assessment  as  a “single-moment-in-time  test  at  the  end  of  instruction” 
(Wiggins  & McTighe,  1998,  p.  12).  As  suggested  by  Wiggins  and  McTighe’s  Backward  Design  Model,  our  model 
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encourages  beginning  teachers  to  collect  multiple  and  varied  pieces  of  evidence  over  time  in  order  to  document 
students’  ongoing  inquiry,  rethinking  and  growth.  As  the  teacher  designs  the  activities  and  plans  for  integrating 
technology  and/or  other  materials  into  the  lesson,  consideration  is  given  to  creating  instruction  and  learning 
opportunities  that  recognize  students’  differing  cultural  capital,  promotes  critical  thinking  and  problem  solving,  and 
uses  technology  as  a tool  to  support  meaningful  learning.  Students  could  be  given  options  for  the  format  such  as  a 
newspaper,  short  story,  or  visual  presentation.  A variety  of  software  and  could  be  used  to  create  the  final  products. 
The  novice  teachers  would  plan  to  conduct  process  interviews  with  students  throughout  the  project  in  order  to  gain 
information  about  the  students’  metacognitive  processes  during  the  project’s  process  phase.  Rubrics  would  be  used 
to  assess  the  products  of  student  learning.  During  this  recursive  step,  the  preservice  teachers  might  realize  that  their 
learners  need  other  skills  in  order  to  complete  the  final  product;  thus,  add  additional  objectives  and  modification  of 
the  lesson  activities  would  occur  during  this  stage. 

In  stage  4,  the  teacher  is  asked  to  implement  the  activity  and  assess  student  performance.  These  activities 
generally  occur  simultaneously  as  the  teacher  checks  students’  understanding  and  ability  to  meet  the  lesson 
objectives.  In  the  final  stage,  the  beginning  teachers  evaluate  how  well  they  implemented  the  lesson.  This  is  the 
critical  reflection  piece  of  our  model:  the  metacognitive  component. 

Part  2 of  the  model  focuses  on  the  metacognitive  activities  of  teachers.  This  part  of  the  model  is  designed  to 
help  novice  teachers  develop  more  complex  schemas  for  both  pedagogical  problem  solving  and  pedagogical  content 
knowledge  development.  Based  on  Schon’s  (1983)  idea  of  reflection-in-action,  pedagogical  problem  solving  is  the 
ability  of  the  teacher  to  analyze  a learning  situation  in  progress,  select  appropriate  strategies  or  interventions  and 
make  corrections  while  teaching.  Pedagogical  content  knowledge  (PCK)  is  the  teachers’  ability  to  identify  learning 
difficulties  and  students’  misconception  combined  with  the  fluidity  to  transform  subject  matter  using  “the  most 
powerful  analogies,  illustrations,  examples,  explanations,  and  demonstrations — in  a word,  the  ways  of  representing 
and  formulating  the  subject  that  makes  it  comprehensible  for  others”  (Shulman,  1986,  p.  9).  The  concept  of  schema 
is  fundamental  to  cognitive  theories  of  representation.  These  theories  seem  to  agree  that  a schema  has  the  following 
characteristics:  a)  it  is  an  organized  structure  that  exists  in  memory,  and  in  total,  contains  the  sum  of  one’s 
knowledge  of  the  world;  b)  it  exists  at  a higher  level  of  generality,  or  abstraction  than  one’s  immediate  experience 
with  the  world  (Paivio,  1974  as  cited  by  Winn,  2004);  c)  it  provides  a context  for  interpreting  new  knowledge  as 
well  as  the  structure  to  hold  knowledge;  and  d)  it  is  dynamic  and  can  be  changed  as  a result  of  general  experience  or 
through  instruction.  As  novice  teachers  learn  through  daily  experiences  and  instructional  intervention,  their 
memories  and  understandings  of  the  world  change  (Winn,  2004).  Based  upon  the  definition  of  schema  as  a dynamic 
construction,  we  propose  to  facilitate  the  continuous  development  of  preservice  teachers’  schema  regarding  effective 
teaching  and  learning. 

Thus,  in  this  stage  of  CITIS  preservice  teachers  are  scaffolded  through  a reflective  process  in  which  they 
consider  four  elements:  student  data,  experiential  data,  cultural  awareness  and  their  mental  models  of  teaching.  First 
the  preservice  teachers  make  meaning  of  the  students’  performance  derived  from  both  informal  and  formal 
assessments  given  during  this  lesson  and  built  upon  it  from  previous  lessons.  Next,  the  preservice  teachers  are  asked 
to  reflect  on  the  experiential  data  derived  from  the  lesson  they  have  recently  designed  and  implemented  in  order  to 
select  Critical  Incidents  (Tripp,  1993)  that  impacted  their  students’  learning.  Tripp  (1993)  suggests  that  teachers 
identify  or  “create”  Critical  Incidents  based  upon  value  judgments:  “Incidents  happen,  but  critical  incidents  are 
produced  by  the  way  we  look  at  a situation”  (p.8).  Critical  Incidents  provide  preservice  teachers  a venue  for  deeper 
and  more  profound  levels  of  reflection  as  well  as  a means  to  challenge  their  schema  (Griffin,  2003;  Hamlin,  2004). 
By  “a  deeper  level  of  reflection”,  we  are  referring  to  not  only  analysis  and  clarification  of  practices  and 
consequences  (technical),  meanings  and  assumptions  (practical),  but  also  consideration  of  ethical,  moral,  and 
political  issues  (critical)  (Gay  & Kirkland,  2003;  Sparks-Langer,  Simmons,  Pasch,  Colton,  & Starko,  1990;  Van 
Manen,  1977).  Using  Griffin’s  (2003)  framework  for  reflecting  on  the  Critical  Incident,  preservice  teachers 
document  what  occurred  during  the  episode,  how  they  felt  about  it  and  why  certain  events  happened  (telling  it  from 
the  perspective  of  each  participant).  Next  they  classify  aspects  of  teaching,  learning  and  schooling  that  occurred  and 
connect  this  to  educational  theories,  professional  standards,  etc.  The  preservice  teachers  are  then  asked  to  consider 
their  own  cultural  background,  beliefs,  prejudices  and  notions  of  culture  and  how  these  might  have  influenced  the 
lesson  design,  lesson  implementation,  and  their  students’  understandings  and  perceptions.  Next  they  describe  their 
mental  model  of  effective  teaching  and  compare  their  own  practice  to  this.  With  the  insights  gained  from  this 
metacognitive  activity,  the  novice  teacher  is  encouraged  to  make  modifications  to  the  current  activity  and  future 
lessons,  thus  moving  from  reflection  to  action  (Killion  & Todnem,  1991). 
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Electronic  Support 

Two  tools  have  been  designed  to  be  included  in  an  electronic  support  system  for  urban  preservice  teachers 
based  on  the  CITIS  model,  careful  analysis,  and  review  of  current  literature.  The  Lesson  Planner  will  provide  users 
with  an  activity  design  “wizard”  that  would  guide  the  novice  teacher  through  planning  an  activity  using  a series  of 
textual  prompts.  By  wizard,  we  mean  an  electronic  device  that  would  guide  the  user  through  a given  process  through 
a series  of  textual  and/or  visual  prompts.  An  example  of  this  might  be  software  that  assists  users  in  completing  their 
tax  returns.  The  textual  prompts  would  be  guided  by  stages  of  lesson  design  as  described  in  the  CITIs  model.  We 
have  also  considered  using  pedagogical  agents  as  guides  and/or  information  sources  (for  more  on  using  pedagogical 
agents,  see  Baylor,  Kisantas,  Chun,  2001;  Kisantas,  Baylor,  Hu,  2001).  The  Video  Window  will  be  a mechanism 
through  which  users  can  upload  video  footage  of  themselves  teaching,  edit  the  footage,  and  reflect  on  their  teaching 
experience  - while  taking  advantage  of  the  immediacy  of  the  digital  video  media.  The  video  window  is  essentially  a 
tool  that  facilitates  novice  teachers’  progression  through  the  metacognitive  components  of  CITIs.  For  example  the 
video  can  be  used  to  assist  them  in  reflecting  on  critical  incidents  after  the  real-time  teaching  moment  (Griffin,  2003; 
Hamlin,  2004;  Killion  & Todnem,  1991;  Tripp,  1993). 

It  is  hoped  that  the  mechanism  created  by  combining  the  Lesson  Planner  and  Video  Window  tools  would 
facilitate  an  iterative  process  that  should  help  develop  reform-based  teaching  habits  and  prepare  teachers  for 
working  with  digital  technology  and  contextual  diversity  of  the  modern  classroom.  Student  use  of  these  two  tools 
could  help  create  powerful,  realistic  teaching  cases  to  be  shared  with  peers  or  stored  in  a knowledge  management 
system  for  future  use. 
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Digital  Primary  Sources  for  Classroom  Instruction:  What  do  teachers  say? 
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Introduction. 

In  history  or  social  studies  classrooms,  teachers  emphasize  the  importance  of  how  people,  geography,  and 
economics  are  interconnected  to  bring  about  the  events  studied  in  the  textbook.  Teachers  of  language  arts  and 
techniques  for  creative  writing  know  that  written  communication  is  greatly  enhanced  through  personal  experience. 
The  curriculum  for  many  school  districts  include  writing  skills  in  “problem-solution”  or  “definition  essays” 
Information  literacy  skills  are  best  taught  using  motivational  forms  of  media  that  touch  the  heart  and  soul  of  a 
student.  Diaries  of  Civil  War  brides,  photos  from  the  Great  Depression,  and  digital  clips  of  music  from  "Negro 
churches"  of  the  19th  century  add  realism  and  pizzazz  to  the  chronological  events  typically  presented  in  school 
classrooms  or  media  centers.  These  and  other  primary  sources  are  being  digitized  and  are  accessible  from  many 
excellent  online  sources.  Digital  primary  source  (DPS)  documents  and  artifacts  are  easily  accessed  through  portals  at 
academic  libraries,  museums,  and  government  agencies.  The  number  of  these  primary  source  websites  is  growing. 

By  searching  the  Internet  Public  Library  (Available:  http://www.ipl.org/),  hundreds  of  educational  sites  can  be 
located.  For  example,  by  using  the  keywords  “digital  primary  sources”  in  the  Education  subject  heading  470  entries 
are  found  using  www.IPL.org  website.  A review  of  the  literature  reveals  that  teachers  and  media  specialists  have 
used  PDS  as  a tool  for  critical  thinking  and  analysis  of  historical  artifacts  (Bennett  & Trofanenko,  2002).  McElmeel 
(2001)  suggests  that  one  of  the  best  uses  of  these  resources  is  for  analytical  comparisons.  PDS  documents  can  be 
used  to  take  the  student  beyond  simple  reading  and  reporting  of  information.  Using  a case  study  at  a nearby  high 
school  and  focus  groups/interviews  from  a representative  sample  of  teachers  across  the  region,  this  researcher  began 
an  investigation  in  how  PSD  can  be  used  to  guide  students  and  teachers  from  simple  fact  gathering  and  reporting  to 
the  applied  use  of  information  to  solve  problems  and  generate  new  ideas  (Loertscher,  2004).  The  purpose  of  this 
study  is  to  investigate  teachers'  understanding  in  the  instructional  use  of  primary  source  documents  and  their 
personal  evaluation  of  a newly  developed  resource  developed  by  a regional  university  in  eastern  North  Carolina. 

Review  of  the  Literature 

Constructivist  classrooms  and  laboratories  are  common  in  K12  schools.  Resource  and  project-based 
learning  are  highly  valued  (Todd,  2005).  Higher-order  thinking  includes  processes  for  content  analysis,  synthesis  of 
new  ideas,  and  evaluation  of  documents,  all  of  which  are  the  valued  outcomes  for  inquiry-based  projects  (Lamb, 
2004)  Primary  source  documents  are  excellent  resources  for  guiding  students  in  constructivist  thinking.  Some  of  the 
benefits  for  using  primary  sources  documents  include  historical  thinking  [processes],  point  of  view,  and  identifying 
bias  (Friedman,  2005).  In  addition,  students  can  be  taught  to  become  active  investigators  who  will  gather  evidence 
that  can  be  used  to  write  their  own  history  book  (VanFossen  & Shiveley,  2000).  Students  are  motivated  to  read  first 
person  accounts,  especially  when  the  author  is  a young  person  who  may  have  lived  in  a different  time  period 
(Barton,  2005).  For  example,  the  strong  and  sustained  interest  in  the  journals  of  Anne  Frank  is  an  indication  that 
young  people  relate  to  the  personal  writings  of  peers  who  have  lived  in  different  times  and  cultures  (Anne  Frank 
House,  2005). 

Currently,  the  most  expansive  project  for  PSD  is  the  “American  Memory  Project”  provided  by  the  Library 
of  Congress  (Balas,  1 999).  Over  four  million  objects  are  archived  on  the  project's  server.  Instructional  anchors 
guide  the  teacher  through  basic  information  related  to  use  of  digital  resources,  best  practices  for  the  classroom,  and 
methods  for  integrating  into  specific  content  areas.  “Documenting  the  American  South”  is  a project  sponsored  by  the 
University  Library  at  the  University  of  North  Carolina  (University  of  North  Carolina,  2005).  Documents  and 
artifacts  available  on  this  website  are  selected  by  the  UNC  Library  editorial  board.  This  valuable  resource  for 
teachers  includes  an  expansive  database  with  artifacts  related  to  the  southern  region  of  the  United  States.  The 
Chicago  Historical  Society,  the  Gilder  Lehrman  Institute  of  American  History,  and  others  are  working 
collaboratively  with  the  University  of  Houston  to  develop  “Digital  History”  (University  of  Houston,  2005),  a richly 
endowed  interactive  multimedia  history  of  the  United  States.  Unique  features  of  “Digital  History”  include  an  online 
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textbook  with  interactive  timelines,  guided  readings,  and  an  online  expert  known  as  “Hyperhistorian”.  Hundreds  of 
primary  source  documents  and  objects  are  listed  in  the  expertly  designed  pathfinders  within  this  website. 

In  2003,  Joyner  Library  at  East  Carolina  University  received  a Library  Services  and  Technology  Act 
(LSTA)  grant  (North  Carolina  State  Library,  available  online)  to  supply  funding  needed  for  the  “North  Carolina 
Digital  History  and  Fiction”  (Joyner  Library,  2005)  project.  Working  in  partnership  with  three  regional  museums, 
the  website  provides  hundreds  of  digitized  documents  related  to  the  eastern  area  of  this  state  (available  online 
http://www.lib.ecu.edu/ncc/historyfiction/  ).  Approximately  200  texts  have  been  digitized  pertaining  to  the  history  of 
29  eastern  North  Carolina  counties.  The  documents,  dating  from  the  early  19th  century,  are  extremely  rare,  fragile, 
and  not  accessible  through  normal  library  circulation.  Joyner  and  its  partners  are  making  available  over  20,000 
additional  pages  of  historic  texts  related  to  eastern  North  Carolina.  An  interesting  collaboration  between,  the 
“Tobacco  Farm  Life,  Country  Doctor  Museum,”  and  “Historic  Hope  Plantation”  has  yielded  a rich  display  of 
artifacts,  objects,  video,  and  audio  files  that  are  uniquely  important  to  eastern  North  Carolina  history  and  culture. 

The  URL’s  for  each  of  these  museums  is  included  in  the  references  for  this  article.  In  addition,  there  are  links  to  NC 
ECHO  (ECHO,  2005)  which  links  to  Cultural  Heritage  sites  located  across  the  United  States.  These,  and  other 
online  resources,  provide  a large  collection  of  digital  artifacts  that  include  metadata  for  descriptions  and 
recommended  use  of  the  resource. 


Research  Method 

During  the  fall  and  spring  semesters  for  2004-2005,  graduate  students  enrolled  in  media  and  technology 
courses  have  been  given  access  to  the  North  Carolina  Digital  History  and  Fiction  Project.  Their  discussion  threads 
and  interviews  have  provided  insight  into  the  usability  of  the  resources  on  the  ECU  digital  project.  In  spring  of  2005, 
a series  of  focus  groups  were  conducted  in  several  locations  in  the  eastern  region  of  North  Carolina.  Teachers, 
media,  and  technology  specialists  from  elementary,  middle,  and  high  school  were  randomly  selected  from  a list 
replying  to  an  invitation  to  review  and  evaluate  the  North  Carolina  Digital  History  and  Fiction  Library.  In  order  to 
achieve  representative  samples  from  the  K12  schools,  the  participants  were  selected  based  on  grade  level  and 
content  area.  Participants  were  compensated  with  a small  stipend  and  a light  supper  meal.  James ville  High  School 
in  the  "down  east"  country  of  the  state  participated  in  a case  study  use  of  primary  sources  provided  by  the 
researcher.  These  included  The  “American  Memory  Project,”  “Documenting  the  American  South”  and  the  “North 
Carolina  Digital  History  and  Fiction  Library”  project.  The  media  coordinator  and  classroom  teachers  worked 
collaboratively  with  the  university  researcher  to  examine  (1)  student  response  to  the  use  of  digital  primary  sources, 
(2)  efficiency  in  use  of  research  strategies,  and  (3)  quality  of  writing  for  the  final  research  paper.  By  using  the 
archived  stories,  interviews,  diaries,  and  other  documents  available  through  digital  primary  sources,  students  have 
authentic  artifacts  for  analysis  using  an  efficient  method  of  access. 

Results  and  Discussion 

Discussion  Threads 

In  spring  and  fall  of  2005,  graduate  students  in  EDTC  6035  Integrating  Computers  in  Social  Studies, 
Language  Arts,  and  Information  Literacy  Skills  were  asked  to  evaluate  the  contents  and  design  of  the  North  Carolina 
Digital  History  and  Fiction  website.  The  enrollments  (n=  52)  for  this  course  consist  of  classroom  teachers  - grades 
K-12,  library  media  specialists,  and  technology  facilitators  in  N.C.  public  schools.  Graduate  student  responses  were 
framed  within  the  following  contexts-  1)  What  is  most  useful;  2)  What  is  least  useful;  and  3)  Instructional  use  for  the 
website. 
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Table  1.  Summary  of  pro’s  and  con’s  for  North  Carolina  Digital  History  and  Fiction  Library  as  evaluated  by 
teachers. 


Most  Useful 

Not  Useful 

Instructional  Uses 

Could  be  used  with 
Webquests 

Video  clips  take  long  time 
to  download 

Useful  to  teach  research  skills  using 
such  models  as  Big6  or  I SEARCH 

Maps 

Search  option  not 
efficient 

Complete  a research  project,  copy 
public  domain  images  and  insert  in 
PowerPoint  for  class  presentations 

Detailed  Information 
that  could  be  used  to 
build  database  files  and 
spreadsheet  activities; 
graphing  activities 

Needs  a better  navigation 
system.  Needs  more  cross 
references  among 
resources 

Research  personal  names  in  home 
county 

NC  ECHO  with  cultural 
heritage  information 

Reading  levels  too  high 
for  elementary  and  middle 
school  students 

Map  skill  using  the  travel  maps 
available  with  each  of  the  29  counties 

Museum  Partners 

Homepage  needs  more 
information  about  purpose 
of  the  site 

Use  the  museum  images  as  advance 
organizers  before  taking  a field  trip 

Easy  navigation 

Homepage  needs  more 
color,  especially  for 
younger  students 

Use  the  museum  images  as  a follow- 
up activity  after  a live  filed  trip 

Tutorial  for  Search 
Operators 

All  laptops  trying  to 
access  the  same  site  at  the 
same  time. . .too  slow 

Visualization  projects  for  life  in  the 
past 

Images  and  objects 
from  the  museums 

Needs  more  alignment 
with  state  standards  and 
state  curriculum 

Students  and  parents  visit  the  sites. 
Parents  and  grandparents  share  oral 
histories  with  students 

Access  to  rare  materials 

Pirate  projects  - what  were  they 
really  like? 

Clear  uncluttered  pages 

Students  read  an  excerpt  from  a 
primary  source  and  answer  a 
question  or  situation  about  it. . . 

Interactive  Map  on 
homepage 

Art  lesson  using  paper  mache  or 
pottery  to  recreate  artifacts  on 
museum  site. 

Image  and  text  version 
of  the  books 

Graduate  student  evaluators  clearly  were  supportive  of  the  Joyner  website.  The  strongest  response  came  from  that 
student  who  identified  the  website  as  a useful  resource  for  local  histories  and  for  teaching  basic  facts  about  North 
Carolina  history.  Nearly  every  evaluator  commented  on  the  value  of  the  website  for  local  history  and  for  fostering 
pride  in  one’s  birthplace  and  personal  heritage.  Many  of  the  evaluators  commented  on  their  own  county  region  first, 
before  any  mention  of  other  resources  available  through  the  website.  Many  of  the  evaluators  live  in  a region  of  the 
state  west  of  the  29  county  region  that  is  the  focus  of  the  website.  These  teachers  all  wanted  to  know  when  the 
remaining  7 1 counties  would  be  added. 

Focus  Group  Interviews. 

An  analysis  of  the  transcripts  from  the  focus  groups  provided  very  useful  formative  information  for  the 
design  and  development  team  of  the  project.  Because  the  focus  groups  were  conducted  earlier  in  the  year  (2004), 
many  of  the  interactive  features  for  the  website  needed  reevaluation  and  revision.  Comments  from  the  focus  groups 
greatly  facilitated  this  process.  The  most  valuable  information  gained  form  the  focus  groups  included  comments  on 
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both  the  design  of  the  website  and  the  informational  contents.  The  strongest  recommendation  was  to  add  links  to 
resources  for  all  of  the  100  counties  in  North  Carolina.  Students  and  teachers  are  “naturally  going  to  search  the 
contents  of  their  home  county  first!”  Many  of  the  focus  group  evaluators  commented  on  design  of  the  homepage. 
Suggestions  were  made  to  add  more  color,  thus  making  the  website  more  inviting  for  elementary  and  middle  school 
students.  Others  suggested  that  links  be  added  for  “kids’  page”  and  a teacher  resource  page.  The  most  common 
recommendation  for  content  was  “add  a topics  page”  with  links  to  specific  topics  related  to  the  North  Carolina 
Standard  Course  of  Study.  Teachers  also  liked  the  idea  of  linking  suggested  lesson  plans  to  specific  student 
competencies  with  in  the  state  curriculum.  Many  of  the  teachers  suggested  adding  shorter  lesson  activities  rather 
than  a full  and  formal  lesson  plan.  Teachers,  they  commented,  rarely  use  a full  lesson  plan  as  written,  but  instead, 
pull  selected  activities  for  use  in  their  classrooms.  Teachers  also  liked  maps,  but  found  little  practical  use  for  the  soil 
sample  survey  maps.  Many  remarked  on  the  excellent  selection  of  fictional  works  dating  from  the  1800’s.  Many  of 
the  high  school  teachers  commented  on  how  these  might  be  used  in  their  literature  classes.  Middle  school  and  high 
school  teachers  commented  on  the  readability  of  most  fiction  and  nonfiction  works.  These  would  be  “too  hard  to 
read”  for  their  students.  Some  suggested  that  excerpts  from  selected  readings  could  be  used  to  teach  concepts  and 
skills  for  “main  idea”,  “point  of  view”,  and  other  reading  comprehension  skills  that  are  included  in  standardized 
tests.  Several  teachers  liked  the  basic  information  listed  on  each  County  homepage.  Though  basic  factual 
information  is  mainly  used  for  recall  questions,  it  is  useful  for  standardized  tests  that  are  typically  administered  in 
K12  public  education.  In  addition,  the  almanac  information  is  useful  as  a springboard  for  problem  solving  and  other 
research  topics  requiring  baseline  information.  For  example,  one  teacher  suggested  using  the  information  for 
building  databases  and  comparison  problems.  Another  teacher  suggested  using  the  population  data  to  show  trends  in 
growth  by  region  and  county.  To  quote  one  high  school  social  studies  teacher: 

My  problem,  and  every  history  teacher’ s problem  in  this  state,  is  that  we  are  driven  by  the  End  of  Course 
test.  I can  tell  you  right  now,  we  would  not  have  the  time  for  students  to  look  up  who  opposed  slavery  in 
North  Carolina.  We  would  tell  them,  the  Quakers,  then  move  on.  . . We  might  discuss  the  Civil  War  and  I 
could  use  a Civil  War  poem,  but  I wouldn’t  expect  the  students  to  go  out  and  find  the  poems  themselves. 
Other  teachers  commented  on  advantages  that  could  be  used  to  support  the  End  of  Course  tests. 

It  would  be  good  if  10th  grade  writing  teachers  could  search  with  the  keywords  that  lead  to  some  quotes  or 
speeches  from  Eastern  North  Carolina. . .keywords  like  heroism  or  justice,  and  then  they  could  ask  the 
students  to  respond  to  the  quotes  in  their  essays.  . . teachers  need  access  to  famous  speeches  and 
documents  which  cover  particular  themes  so  that  students  can  practice  writing  for  the  (EOC)  test. 

Case  Study. 

Students  at  Jamesville  high  school  (Dotson  & Grimes,  2005)  were  instructed  to  research  a topic  and  locate 
information  that  would  provide  answers  or  a conclusion  to  the  focused  research  questions.  Results  from  their 
research  of  primary  sources  were  presented  electronically  by  using  PowerPoint  slides.  At  the  conclusion  of  the 
project,  each  student  completed  a reflective  response  sheet.  Students  responded  to  questions  related  to  their  interest 
in  digital  primary  sources,  the  most  helpful  websites,  usability  of  the  websites,  what  significant  things  were  learned, 
satisfaction  levels  for  the  use  of  the  technology,  and  what  parts  of  the  project  caused  frustration  and  anxiety. 
Students  were  also  questioned  on  their  understanding  of  primary  sources  and  the  benefits  for  using  this  kind  of 
resource  for  research.  Comments  from  the  reflective  surveys  were  summarized  and  analyzed  for  patterns  that  might 
indicate  a major  theme  underlying  attitudes  and  understanding  of  primary  source  documents.  . As  can  be  seen  in 
Table  2.  comments  fall  into  two  categories.  About  half  the  students  found  the  sources  helpful  and  enjoyed  the 
project.  In  contrast,  several  students  reported  frustration  and  lack  of  satisfaction. 
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Table  2.  Digital  Primary  Sources:  What  was  helpful  and  what  was  frustrating  for  secondary  school 

students. 


What  was  helpful 

What  was  frustrating 

The  document  helped  me  write  my  focus 
question  [note:  student  research  strategy  is  in 
reverse  order] 

"can't  find  anything"  comments=  8 total 

I read,  in  depth,  some  documents  and  made 
inferences. 

navigation  problems;  dead  links=  7 total 

More  interesting  and  more  useful  [than 
secondary] 

Hard  work,  had  to  dig  into  the  information  to 
find  answers 

"Enjoyed  going  into  the  school  library  to  work" 
comments  = 4 total 

Sometimes  the  websites  were  confusing  and 
hard  to  use  . . .confusing  and  outdated. . . . 

My  focus  question  helped  me  stay  on  topic  and 
not  include  information  that  did  not  support  or 
relate  to  my  topic. 

Primary  research  is  much  harder  than  using 
[just]  any  source  available. 

. . .much  more  accurate  because  it  was  written  by 
the  people  of  that  time  period. 

. . .time  consuming,  missed  being  in  the 
classroom. . . time  consuming 

Focus  question  narrowed  my  search  and  it  was 
less  time  consuming  for  the  research. 

. . .just  did  searching  and  wrote  my  focus 
question  at  the  end. . . .[reverse  order] 

I learned  that  people  living  in  different  places 
had  different  and  like  experiences,  but  also 
people  felt  different  about  things  like  the  war. . . 

. . .learned  it  is  difficult  to  do  a presentation 
from  newspapers  and  diaries. . . 

. . .learned  to  dig  deep  into  a database 

. . .learned  that  some  views  in  modern  times  and 
history  disagree  with  the  views  of  people  during 
that  time... 

You  get  first  person  views  of  your  research 
. . .you  learned  about  how  people  reacted  to  a 
historic  event.... 

Clearly,  students  need  instruction  in  how  to  use  primary  source  documents  within  a particular  context. 
Students  in  this  case  study  were  able  to  describe  the  characteristics  for  a variety  of  primary  documents,  however,  it 
was  not  until  a second  project  was  planned  that  students  began  to  formulate  questions  that  used  point-of-view, 
comparisons  problems,  and  bias  in  primary  documents.  Leslie’s  project,  on  the  other  hand,  showed  great  potential 
for  research  with  point-of-view.  She  demonstrated  understanding  of  primary  sources  as  biased  and  that  the 
researcher  must  "write  history"  based  on  comparison  of  a variety  of  documents.  In  response  to  reflective  question, 
"What  was  the  most  significant  piece  of  information  you  learned  in  doing  this  project,"  she  responded,  “Opinions 
vary  greatly  depending  on  what  you  are  reading  and  who  you  are  reading.  Also,  since  you  are  reading  opinions  don’t 
take  everything  as  fact.”  When  students  were  asked,  "What  did  you  learn  about  primary  sources  for  doing 
research?... that  primary  sources  we  had  were  not  very  good  at  finding  information,”  strongly  indicates  a need  to 
provide  more  effective  methods  for  instruction  in  the  purpose  and  best  use  of  primary  source  documents. 

Conclusions 

Based  on  the  analyses  of  focus-group  transcripts,  interviews,  discussion  threads,  and  survey  responses,  the 
use  of  digital  primary  sources  can  be  a valuable  resource  for  teachers,  media  and  technology  specialists.  There  are 
three  main  benefits  with  the  use  of  online  primary  sources.  First,  teachers  report  that  the  digital  documents  and 
artifacts  are  motivational  and  engaging  for  students.  Images  of  artifacts,  sound  bytes,  and  interviews  using  streaming 
video  provide  rich  multimedia  environments  for  students.  Secondly,  they  are  a good  tool  for  enhancing  higher  order 
thinking  processes.  Students  have  the  opportunity  to  “write  their  own  history”  through  position  papers,  point-of- 
view,  and  analysis  of  controversial  documents.  Finally,  digital  primary  sources  are  readily  available  on  the  web  and 
accessible  to  any  student  with  Internet  access.  A simple  search  of  the  web  using  any  of  the  popular  search  engines 
will  result  in  several  thousand  valid  educational  websites  housing  the  primary  digital  resources. 


99 


There  are  some  disadvantages.  Websites  with  streaming  audio  and  video  presentations  that  include 
metadata  or  other  supporting  information  may  not  be  accessible  to  schools  with  low  bandwidth  or  older  computers. 
Slow  downloads  lead  to  problems  with  classroom  management  and  impatient  students.  Understandably,  the  original 
documents  were  usually  written  for  adults,  thus  reading  levels  may  not  be  suitable  for  younger  students.  When 
planning  lessons  for  middle  school  and  children  in  older  elementary  grades,  excerpts  from  documents  should  be 
selected  in  advance  and  presented  as  a whole  class  reading  opportunity  and  discussion.  When  using  primary  source 
documents  for  a traditional  “research  paper”,  teachers  should  review  and  evaluate  several  sources  prior  to  classroom 
instruction  and  research  activities.  Selected  resources  should  be  bookmarked  and  a pathfinder  (Kuntz,  2003) 
provided  to  eliminate  aimless  wandering  on  the  web  and  loss  of  valuable  time  for  thinking  and  problem  solving 
discussions. 

Digital  primary  sources  provide  documents  and  artifacts  that  bring  history  and  culture  to  a new  dimension 
for  both  teachers  and  students.  Priceless  resources  in  the  form  of  fiction  and  nonfiction  can  be  read  by  any  student 
without  the  necessity  of  traveling  to  an  archived  collection  in  a distant  library  or  museum.  Most  importantly,  this 
paper  suggests  that  students  have  the  potential  for  reading  first  hand  accounts  and  drawing  their  own  conclusions 
about  controversial  topics.  Students  no  longer  must  be  entirely  dependent  on  the  historical  reports  by  secondary 
sources  but  have  opportunity  to  think  as  an  historian  does,  piecing  together  the  story  from  a variety  of  sources  and 
personal  accounts. 
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Abstract 

Human  Performance  Technology  (HPT)  provides  interventions  for  improving  performance  of  humans, 
groups  and  organizations  in  variety  of  settings.  Effective  evaluation  strategies  are  needed  to  justify  and  improve 
these  interventions.  Kirkpatrick’ s framework  is  widely  utilized  to  evaluate  training  as  well  as  non-training  HPT 
interventions  in  business  setting.  However,  the  distinction  between  the  natures  of  training  and  non-training  solutions 
in  HPT  requires  a new  evaluation  model.  This  paper,  first,  has  analyzed  strengths,  weaknesses  and  utilization 
aspects  of  Kirkpatrick’ s framework  in  corporate  settings,  second,  has  examined  alternative  evaluation  models  and 
finally,  has  proposed  the  Dimensions  and  Systems  model  for  evaluating  HPT  interventions. 

Introduction 

Performance  Technology  (PT)  is  a field  committed  to  enhancing  the  performance  of  human  beings  and 
organizations  (Van  Tiem,  Moseley,  & Dessinger,  2000).  In  order  to  validate  the  effectiveness  of  PT  interventions,  an 
evaluation  process  is  needed  (Rosenberg,  Coscarelli,  & Hutchinson,  1999).  In  1959,  Kirkpatrick  published  a four- 
step  evaluation  model  consisting  of  reaction,  learning,  behavior  and  results  to  evaluate  corporate  training  programs 
(Kirkpatrick,  1996).  Since  then,  the  model  has  become  a core  for  several  evaluation  models  (Goldwasser,  2001). 
Kirkpatrick’s  framework  is  not  only  used  for  evaluating  training  programs,  but  also  for  evaluating  performance 
improvement  interventions  (Klein,  2002).  However,  training  is  not  the  only  solution  to  performance  problems,  so 
Kirkpatrick’s  framework  may  not  be  sufficient  to  evaluate  the  effectiveness  of  PT  interventions.  This  paper  aims  to 
provide  an  expanded  view  of  Kirkpatrick’s  framework  to  meet  the  need  for  a complete  model  for  evaluating  PT 
interventions. 

Human  Performance  Technology  is  a systemic  and  systematic  way  of  applying  scientific  and  practical 
knowledge  about  human,  group  and  organizational  behavior  to  improve  human  performance  and  achievement 
economically  and  effectively  by  using  a wide  range  of  interventions  drawn  from  several  disciplines  (Rosenberg, 
1996;  Rossett,  1996;  Stolovitch  & Keeps,  1999).  The  field  is  usually  referred  to  as  Performance  Technology  (PT) 
with  the  human  prefix  implied  (Rosenberg,  1996).  The  operational  dimensions  of  PT  are  divided  into  human, 
workplace  and  work  (Rosenberg,  1996).  PT  practitioners  employ  a wide  range  of  solutions  to  any  given 
performance  problem  including,  but  not  constrained  to,  training  (Rosenberg,  1988). 

Evaluation  is  a process  of  making  judgments  about  the  worth  of  observed  phenomena  based  on  collected 
data  (Shrock  & Coscarelli,  2000).  In  the  PT  field,  formative  evaluation  is  conducted  during  the  intervention 
development  process  and  the  summative  evaluation  process  is  conducted  after  intervention  implementation 
(Thiagarajan,  1991).  Summative  evaluation  has  three  purposes,  to  justify  existence  of  the  intervention,  to  improve 
the  intervention  for  possible  future  implementation,  to  determine  whether  to  continue  or  stop  the  intervention 
(Kirkpatrick,  1998).  This  paper  will  focus  on  summative  evaluation. 

Significant  resources  are  spent  on  training  and  non-training  interventions  to  increase  human  performance 
(Lee  & Pershing,  2002).  As  a result,  PT  practitioners  are  expected  to  make  contributions  to  the  organizations’ 
bottom  line  (Phillips,  1991;  Robinson  & Robinson,  1989;  Rosenberg,  1996).  In  order  to  determine  effectiveness  and 
justify  PT  interventions,  evaluation  is  needed.  Since  the  literature  has  not  presented  a comprehensive  model  to 
evaluate  the  effectiveness  of  PT  interventions  (Holton,  1996;  Lee  & Pershing,  2002;  Sleezer,  Zhang,  Gradous,  & 
Maile,  1998).  Generally,  PT  practitioners  utilize  Kirkpatrick’s  four-level  framework  for  evaluating  PT  interventions. 
The  reasons  for  this  situation  stem  from  the  origins  of  PT  and  current  PT  practices. 

Origins  of  PT  and  Training’s  Place 

As  implied  in  the  definition  of  PT  and  claimed  by  several  authors,  training  is  not  the  only  solution  to 
performance  problems.  As  a field,  PT  evolved  from  instructional  systems,  systems  thinking,  informatics, 
ergonomics  and  psychometrics  (Rosenberg  et  al.,  1999).  Historically,  the  roots  of  PT  are  deeply  embedded  in 
Instructional  Systems  Design  (ISD)  in  terms  of  concepts,  principles  and  theories  (Reiser,  1987). 
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Additionally,  many  practitioners  have  stated  that  training  is  the  least  effective  approach  to  performance  problems 
(Gordon  & Zemke,  2000;  Zemke  & Rossett,  2002).  PT  practitioners  are  therefore  shifting  from  training  to  other 
performance  improvement  solutions  (Carr  & Totzke,  1995;  Guerra,  2001).  However,  this  transition  is  a slow  one. 
According  to  Rossett  and  Tobias  (1999),  40  percent  of  practitioners  do  not  conduct  any  needs  assessment  and 
implement  training  as  the  only  solution  to  the  performance  problems  in  their  organizations.  Furthermore,  Klein 
(2002)  conducted  a literature  review  of  PT  interventions  published  in  Performance  Improvement  Quarterly,  the 
premier  journal  of  the  PT  field,  and  his  findings  show  that  the  prevailing  type  of  intervention  is  several  forms  of 
training  in  performance  improvement.  Consequently,  Kirkpatrick’ s framework  is  the  only  tool  to  evaluate  these 
interventions.  The  framework  suggests  four  levels  (Kirkpatrick,  1998): 

Reaction : “How  those  who  participate  to  the  program  react  to  it. . .a  measure  of  costumer  satisfaction.”  (p. 
19) 

Learning : “The  extent  to  which  participants  change  attitudes,  improve  skills  as  a result  of  attending  the 
program.”  (p.  20) 

Behavior.  “The  extent  to  which  change  in  behavior  has  occurred  because  the  participant  attended  the 
training  program.”  (p.21) 

Results'.  “The  final  results  (low  absenteeism,  increased  profit,  low  turn  over,  tangible  results)  that  occurred 
because  the  participants  attended  the  [training]  program.”  (p.23) 

The  Philosophy  and  Utilization  of  the  Framework 

Although  it  was  introduced  in  1959,  the  framework  is  still  in  use  and  has  served  as  source  of  inspiration  for 
several  approaches  to  evaluating  training  programs  (Kaufman  & Keller,  1994;  Molenda,  Pershing,  & Reigeluth, 
1996;  Phillips,  1996),  human  resources  development  (Dye,  2001;  Holton,  1996)  and  teacher  professional 
development  programs  (Guskey,  2000).  However,  its  name  has  evolved  from  four- step  to  four-level  (Kirkpatrick, 
1998)  and  there  is  an  ongoing  controversy  about  whether  it  is  merely  taxonomy  or  a model  (Dye,  2001;  Holton, 
1996)  and  a prescriptive  or  descriptive  framework  (Lewis,  1996;  Twitchell,  Holton,  & Trott,  2000).  For  the  purpose 
of  consistency,  throughout  this  paper,  Kirkpatrick’ s evaluation  approach  will  be  referred  to  as  a framework. 

Why  is  the  framework  so  popular? 

The  reasons,  for  which  the  framework  is  widely  used,  cited,  modified  and  attacked,  are  the  philosophy 
underneath  it,  its  strengths  and  shortcomings.  Drawing  from  Kirkpatrick’s  articles  and  book  written  on  the  four-level 
evaluation  framework,  the  ideas  below  can  be  identified  as  philosophy  of  the  framework: 

1.  Only  for  training:  The  framework  aims  at  evaluating  training  programs  only  (Kirkpatrick,  1998). 

2.  Evaluation  as  complete  a four-level:  According  to  Kirkpatrick  (1996),  partial  application  of  the  four- 
level  evaluation  does  not  complete  the  evaluation  process.  In  order  to  provide  useful  information  about  the 
effectiveness,  a training  program  all  four  levels  should  be  applied  in  sequence. 

3.  Simplicity  and  practicality:  According  to  Kirkpatrick  (1996),  many  practitioners  are  not  interested  in 
complex  and  scholarly  evaluation  methods,  they  seek  for  a model  that  they  can  understand  and  apply. 

4.  Evidence  rather  than  proof:  In  connection  with  the  third  point,  Kirkpatrick  makes  a distinction  between 
proof  and  evidence  when  conducting  level  3 and  level  4 (Kirkpatrick,  1977;  Kirkpatrick,  1998).  He  argues  that  proof 
is  difficult  to  obtain  at  levels  3 and  4,  so  practitioners  should  collect  evidence  about  the  contribution  of  training. 

The  framework’s  philosophy  creates  its  strengths,  below  are  the  identified  strengths  of  the  framework  in  its 
practical  application  as  reviewed  in  the  literature. 

Simplicity  and  elegance : The  most  important  strength  of  the  approach  is  that  it  provides  a simple  and 
applicable  evaluation  model  for  training  practitioners  (Alliger  & Janak,  1989). 

Completeness:  Kirkpatrick  drew  on  all  evaluation  methods  practiced  up  until  his  time  and  created  his 
approach  to  evaluation  with  a complete  framework  (Spitzer,  1999). 

A consistent  framework  for  comparison : The  framework  provides  a consistent  template  to  compare  the 
effectiveness  of  training  or  professional  development  programs  across  industries  and  organizations.  Using  a 
common  framework  to  evaluate  the  programs  could  help  practitioners  to  decide  which  methods  are  effective  in 
training  and  fit  for  their  context. 

But,  is  the  framework  truly  popular? 

Although  several  authors  and  practitioners  claimed  wide  use  of  the  framework  consistent  with  its  strengths, 
research  from  the  literature  does  not  reflect  these  claims.  Studies  from  the  USA  (ASTD,  1996;  Moller  & Mallin, 
1996;  Twitchell  et  al.,  2000),  Canada  (Blanchard,  Thacker,  & Way,  2000),  Britain  (Bramley  & Kitson,  1994)  and 
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Kuwait  (Al-Athari  & Zairi,  2002)  show  that  most  evaluation  activities  are  conducted  at  level  1 and  level  2 and 
spurned  at  levels  3 and  4,  which  makes  partial  use  of  the  framework  widespread.  The  low  utilization  of  level  3 and 
level  4 is  not  a new  issue  (Twitchell  et  al.,  2000).  As  cited  in  Twitchell  et  al.  (2000),  Catalanello  and  Kirkpatrick 
(1968)  published  similar  results  in  the  past. 

Why  were  Level  3 and  Level  4 applied  rarely? 

Regardless  of  country,  industry  and  time,  level  3 and  level  4 evaluations  are  rarely  conducted.  Eight 
answers  to  the  question;  “what  are  the  reasons  for  not  applying  level  3 and  level  4?”  come  from  research  studies  and 
the  experiences  of  prominent  training  practitioners. 

No  need  to  conduct  evaluation  at  level  3 and  level  4:  Dixon  (1996)  suggests  that  some  companies  (such  as 
Motorola,  IBM,  Arthur  Andersen,  etc.)  do  not  feel  a need  to  justify  the  existence  of  their  training  departments. 
Moller  and  Mallin  (1996)  and  (Twitchell  et  al.,  (2000)  support  this  point  with  their  findings. 

The  impracticality  of  utilizing  all  levels  for  every  training  program : Geber  (1995)  argues  that  applying 
level  3 and  level  4 to  every  single  training  program  that  a company  offers  would  be  impractical  for  some 
organizations  because  of  the  nature  of  some  programs. 

Lack  of  knowledge  and  skills : Some  of  the  training  practitioners  do  not  have  the  knowledge  and  skills  to 
design  sophisticated  evaluation  methods  and  the  required  metrics  (Twitchell  et  al.,  2000). 

Lack  of  collaboration:  Some  organizations  do  not  have  a culture  of  openness  and  trust  between  employees 
and  management  (Harrell,  2001)  and  some  managers  perceive  data  collection  and  evaluation  as  disruptive  (Moller  & 
Mallin,  1996;  Riley,  Davani,  Chason,  & Findley,  2002). 

Lack  of  administrative  support : Riley  et  al.  (2002)  listed  stakeholders’  inadequate  understanding  about  the 
importance  of  level  3 evaluation  as  one  reason.  Additionally,  in  a survey  study,  most  of  the  practitioners  stated  that 
their  companies  placed  significant  barriers  to  evaluation  at  all  levels  (Moller  & Mallin,  1996). 

Time  and  money  costs:  Geber  (1995)  states  that  “No  one  who  does  level  3 and  level  4 evaluation  pretends 
that  it’s  free”  (p.34),  level  3 and  level  4 are  more  time  consuming  and  cost  more  money  than  level  1 and  level  2 
evaluations. 

Abuse  of  results:  In  a survey  study,  Moller  and  Mallin  (1996)  found  that  most  practitioners  do  not  want  to 
see  negative  evaluation  results  for  a training  program  as  they  may  be  used  to  cause  damage  rather  than 
improvement. 

Access  to  data:  In  connection  with  lack  of  administrative  support,  access  to  pre-intervention  and  post- 
intervention data  is  another  barrier  (Moller  & Mallin,  1996) 

Why  is  Kirkpatrick’s  framework  used  in  PT? 

The  literature  suggests  three  reasons  for  utilizing  Kirkpatrick’s  framework  in  PT  interventions:  training  as 
the  prevailing  intervention,  lack  of  alternatives  and  the  intuitive  power  of  the  model. 

Training  as  the  prevailing  solution 

As  stated  previously,  instructional  systems  design  has  been  the  main  influence  on  PT.  For  example,  the 
early  name  of  the  field’s  professional  organization  was  the  National  Society  for  Programmed  Instruction  and  later 
became  the  National  Society  for  Performance  Improvement.  Furthermore,  journal  names  for  the  field  followed  suit. 
In  terms  of  solutions  to  performance  problems,  although  there  is  a transition  from  training  to  other  interventions, 
training  is  still  the  dominant  intervention  method  for  performance  problems  (Klein,  2002;  Rossett  & Tobias,  1999). 
Consequently,  in  order  to  evaluate  the  effectiveness  of  performance  interventions,  Kirkpatrick’s  framework  is  the 
primary  model  to  be  utilized. 

Lack  of  alternative  evaluation  models 

As  cited  in  Carr  and  Totzke  (1995),  once,  Maslow  stated  “if  all  you  have  a hammer,  you  will  treat  all 
problems  as  nails”  (p.  11).  Lack  of  models  to  evaluate  performance  improvement  interventions  is  a glaring  problem 
and  it  has  been  stated  by  several  authors  (Sleezer  et  al.,  1998).  Other  evaluation  models  such  as  360-degree  feedback 
and  Balanced  Scorecards  might  be  considered  as  alternatives. 

Intuitive  power  of  the  framework 

The  elements  of  Kirkpatrick’ s framework  were  being  utilized  before  Kirkpatrick  published  his  framework, 
however  not  systematically  as  Kirkpatrick  suggested  (Kirkpatrick,  1998;  Spitzer,  1999).  By  providing  this  simple 
and  complete  framework,  Kirkpatrick  showed  how  a training  program’ s outcomes  could  affect  individuals  and 
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organizations  (Spitzer,  1999).  Practitioners’  experiences  suggest  that  the  levels  of  the  framework  can  be  applied  to 
any  type  of  PT  intervention,  which  makes  these  categories  almost  universal  (Edwards,  Scott,  & Raju,  2003). 

Then,  the  model  should  be  adequate? 

Although  Kirkpatrick’s  framework  has  been  utilized  to  evaluate  training  programs,  it  is  not  sufficient  to 
evaluate  general  performance  improvement  interventions  (Lee  & Pershing,  2002;  Sleezer  et  al.,  1998).  Two  main 
reasons  inherent  to  the  framework  can  be  identified  from  the  literature  as  the  framework’s  inadequacy  for  PT. 

The  Multi-  disciplinary  nature  of  PT  interventions 

As  originally  stated  many  times  by  Kirkpatrick  (1998)  and  other  authors,  the  framework  is  designed  for 
evaluating  training  programs.  The  main  focus  of  training  is  teaching  and  learning,  whereas  the  main  focus  of  PT  and 
the  performance  technologist  is  performance  and  accomplishment  (Rossett,  1996).  Performance  is  a combination  of 
several  variables.  Many  authors  have  attempted  to  provide  a comprehensive  taxonomy  for  performance  variables 
(Holton,  1999;  Sleezer  et  al.,  1999).  (Stolovitch  & Keeps,  1999)  categorized  performance  variables  into  two 
categories,  learning  interventions  and  non-learning  interventions.  Performance  is  a function  of  many  variables  and 
training  is  just  one  of  the  interventions  capable  of  increasing  performance  (Brinkerhoff  & Dressier,  2002). 

Inadequacy  to  take  systems  approach 

One  view  of  PT  is  to  see  the  organizations  as  systems  (Sleezer  et  al.,  1999).  A system  is  a set  of  elements 
interacting  in  an  environment  to  achieve  results  (Ackoff,  1994).  An  outcome  in  an  organization  is  a result  of  several 
interactions  of  related  elements  within  an  environment.  At  level  3 and  level  4,  practitioners  actually  are  not  only 
measuring  the  effects  of  training  on  performance  but  also  a large  system  of  organization’s  effects  to  the  performance 
and  results  (Brinkerhoff  & Dressier,  2002).  Kirkpatrick’s  framework  does  not  accommodate  all  the  elements  that 
have  effects  on  outcomes  at  each  level  and  their  interactions  with  each  other  (Holton,  1996). 

Criteria  for  an  Alternative  Model 

It  is  clear  that  Kirkpatrick’s  framework  partially  meets  the  first  set  of  criteria  and  (Holton,  1996;  Lee  & 
Pershing,  2000),  and  a substitute  for  the  framework  is  needed,  but  what  might  be  the  attributes  of  an  ideal  model  for 
evaluating  PT  interventions?  An  integration  of  knowledge  from  the  literature  suggests  a set  of  criteria;  the  serving 
purposes  of  evaluation  in  PT.  An  evaluation  model  for  PT  interventions  should: 
provide  outcomes  to  improve  interventions, 

provide  outcomes  to  validate  if  the  intended  objectives  are  achieved, 

provide  outcomes  to  justify  existence  of  intervention, 

provide  a systemic  view  of  performance  interventions, 

accommodate  multiple  interventions, 

accommodate  individuals  and  organizations, 

be  applicable  by  practitioners. 

Models  at  work  in  organizations 

The  literature  suggests  other  evaluation  models  for  evaluating  performance  of  individuals  and 
organizations,  among  them,  360-degree  feedback  evaluation  (Scott  & London,  2003),  balanced  scorecards  (Kaplan 
& Norton,  1992)  and  Holton’s  model  for  evaluating  HRD  interventions  (Holton,  1996)  are  the  best  known  models. 

360-degree  feedback 

360-degree  feedback  evaluation  is  a technique  for  evaluating  an  employee’s  job  performance  based  on 
feedback  from  everyone  with  whom  the  employee  has  contact.  Its  primary  methodology  is  collecting  data  from 
multiple  sources  in  an  employee's  environment  (Scott  & London,  2003).  The  main  objectives  of  360-degree 
evaluation  are  to  increase  the  objectivity  of  the  performance  assessment,  to  increase  the  self-awareness  of  the 
performer  and  to  stimulate  professional  development  (Alimo-Metcalfe,  1998).  The  main  drawback  of  this  method  is 
that  it  is  an  aggregated  form  of  evaluation.  It  does  not  aim  to  show  the  effects  of  an  intervention  on  performance. 

Balanced  Scorecards 

The  Balanced  Scorecards  model  has  been  specifically  developed  to  measure  and  evaluate  the  linkage 
between  financial  results  and  employee  behavior  at  the  group  and  organization  levels  (Edwards  et  al.,  2003;  Kaplan 
& Norton,  1992).  Simplicity  and  the  least  number  of  performance  and  result  outcome  measures  are  essentials 
(Kaplan  & Norton,  1992).  Although  the  Balanced  Scorecards  method  is  sufficient  method  for  bottom  line,  it  does 
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not  meet  the  first  three  requirements  of  evaluation  in  PT.  Evaluated  results  and  performance  are  presented  in 
aggregated  forms.  It  does  not  discuss  contributions  of  an  intervention  to  performance  and  its  results. 

Holton ’s  HRD  model 

Holton’s  framework  discusses  how  an  intervention  (training  in  his  case)  can  be  evaluated  in  terms  of 
learning,  performance  and  results  perspectives.  He  provided  an  expanded  view  of  Kirkpatrick’s  framework  and 
claimed  that  the  framework’s  level  1,  reaction,  is  not  an  outcome  of  training,  but  rather  a moderator  (Holton,  1996). 

Holton’s  framework  is  an  effort  to  explain  the  influences  on  outcomes  of  an  intervention,  however,  it 
cannot  be  used  in  PT  evaluation  for  three  reasons.  First,  it  is  not  intended  for  all  performance  improvement 
interventions.  As  stated  by  Holton  (1996),  it  is  intended  for  a single  training  intervention.  Second,  it  evaluates 
performance  on  the  individual  level,  while  PT  focuses  on  groups  and  organizations  as  well  as  individuals.  Finally, 
practitioners  in  the  field  need  a simple  and  elegant  model  for  evaluating  interventions. 

All  three  models  above  only  partially  meet  the  criteria  of  an  ideal  evaluation  model  for  PT.  Therefore,  a 
model  which  addresses  the  criteria  for  an  ideal  evaluation  model  and  attributes  of  PT  is  needed. 

Proposing  an  Expanded  View  of  Kirkpatrick’s  Framework 

In  order  to  address  two  sets  of  criteria  presented  in  the  paper,  a three  dimensional  model  with  systems 
theory  is  proposed.  The  model  is  named  Dimensions  and  Systems  since  application  of  the  model  requires,  first, 
selecting  dimensions  and,  second,  configuring  the  systems. 

Dimensions 

In  order  to  address  all  the  elements  in  PT,  a three  dimensional  matrix  is  needed.  Dimensions  of  the  model 
are  variables/interventions,  evaluation  outcomes  and  subjects  of  evaluation.  Figure  1 depicts  each  dimension. 


External  Economics 
External  Governmental 


Figure  1 . Illustration  of  dimensions 


Subjects  of  Evaluation:  Performance  improvement  interventions  can  be  applied  to  individuals,  groups  or 
organizations  (Brethower,  1999).  Furthermore,  these  interventions  may  have  an  effect  on  external  environments 
(Molenda  et  al.,  1996;  Watkins,  Foshay,  & Kaufman,  1998).  Therefore,  five  subjects  of  evaluation  are  purposed; 
individual,  group,  organization,  industry  and  society. 

Evaluation  outcomes : Evaluation  outcomes  of  an  intervention  are  basically  Kirkpatrick’s  framework 
modified  as;  reaction,  internalization,  performance  and  impact.  The  names  have  been  modified  in  order  to  better 
generalize  to  other  interventions. 

Reaction  refers  Kirkpatrick’ s reaction  level  and  provides  feedback  for  further  improvement  in  the 
intervention,  e.g.,  reactions  to  a training  program  are  feedback  for  further  improvements  (Fee  & Pershing,  2002). 


106 


Internalization  refers  to  the  acceptance  or  acquisition  of  intervention.  Evaluating  internalization  is  intended 
to  determine  if  the  intended  intervention  is  absorbed  by  the  subject,  if  the  subject  is  willing  or  able  to  perform  with 
the  intervention  and  if  the  intervention  is  effective. 

Performance  refers  to  performance  under  the  effects  of  the  intervention.  The  aim  of  PT  is  to  improve 
performance.  Since  behavior  is  training  specific  term,  it  is  replaced  with  the  term  performance  to  embrace  all 
interventions  for  all  subjects. 

Impact  refers  to  accomplishments  of  performance.  Impact  can  be  different  for  each  subject.  Kirkpatrick 
(1998)  referred  to  results  as  tangible  improvements  in  the  characteristics  of  organizations  such  as  reduced  turnover, 
absenteeism  and  scrap,  and  increased  profit  and  costumer  satisfaction.  The  term  “impact”  refers  to  results  at  the 
individual,  group  and  organizational  level.  Watkins  et  al.  (1998)  added  one  more  level  to  Kirkpatrick’s  framework, 
societal  contribution.  In  this  model,  impact  outcome  at  the  social  level  refers  to  Watkins  et  al.  (1998)  societal 
contributions  of  intervention. 

Variables/Interventions  elements : The  third  dimension  of  the  model  is  Variables/Interventions  elements, 
which  affect  the  occurrence  of  each  evaluation  outcome  to  some  degree.  Wile  (1996)  proposed  seven  elements  for 
performance  to  occur,  organizational  systems,  incentives,  cognitive  support,  tools,  physical  environment,  skills 
knowledge  and  inherent  ability.  This  taxonomy  provides  a complete  list  of  outcome  elements  for  reaction, 
internalization  and  performance  evaluation  outcomes.  Holton  (1996)  included  external  to  organization  events 
variable  in  his  model  to  show  the  effects  of  external  forces.  In  this  model,  in  order  to  include  the  influences  external 
to  the  organization,  external  economics  and  governmental  elements  are  added.  Variables/Interventions  can  have  two 
roles,  first,  they  are  the  components  of  each  evaluation  outcome  and,  second,  if  they  are  modified  to  improve  the 
performance,  they  become  interventions  whose  effectiveness  is  evaluated. 

Systems 

The  second  part  of  the  model  is  systems.  Systems  thinking  is  a discipline  dedicated  to  seeing  the  whole,  its 
elements  and  the  interrelations  between  them  (Senge,  1990).  Organizations  are  systems  and  when  performance 
technologists  implement  an  intervention,  this  intervention  has  interactions  with  the  existing  elements  of  the 
organizational  systems  (Brathower,  1999).  Systems  thinking  can  help  a performance  technologist  to  see  the 
complexities  in  organizations  and  the  effects  of  their  intervention  (Sleezer  et  al.,  1999).  Basic  system  elements  are 
inputs,  outputs,  process,  feedback  and  environment  (Hutchins,  1996). 

Inputs  are  Variables/Interventions  elements  that  are  listed  above. 

Process  consists  of  the  subject  of  the  evaluation;  individual,  group,  organization,  industry  or  society. 

Outcomes  are  measured  evaluation  outcomes;  reaction,  internalization,  performance  and  impact. 

Surrounding  environment  is  the  environment  in  which  the  system  exists.  In  the  model  the  surrounding 
environment  for  individual  is  the  group.  For  group,  the  surrounding  environment  is  the  organization  and  so  on. 

Feedback  is  any  effect  that  outcomes  produce  and  feed  back  to  the  inputs.  Depending  on  the  intervention, 
feedback  could  be  motivation. 

By  selecting  the  elements  in  dimensions  model,  appropriate  evaluation  systems  are  to  be  set  up. 

Application  of  the  Model 

Selecting  outcomes  to  be  evaluated 

As  the  literature  shows,  not  all  levels  of  Kirkpatrick’s  framework,  especially  level  3 and  level  4 are 
practiced.  Reasons  for  this  situation  vary  from  organization  to  organization.  A decision  matrix  for  the  objectives  of 
the  evaluation  should  be  set  up  to  selects  appropriate  outcomes  to  be  evaluated.  The  items  in  the  following  table  are 
an  integration  of  the  criteria  that  practitioners  use  when  designing  an  evaluation  (Geber,  1995;  Twitchel  et  al.,  2000; 
Moller  and  Mallin,  1996;  Dye,  2001). 


Table  1.  Decision  matrix  for  obtaining  evaluation  outcomes 
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Number  or  people  affected  by  the 
intervention 

Expected  impact  of  intervention 

Expected  cost  of  evaluation 

Access  to  data 

Required  competency/skills  to  obtain 
outcome 

Climate  toward  evaluation 

Decision 

After  deciding  on  the  evaluation  outcome,  a practitioner  needs  to  consider  hierarchical  order  of  outcomes 
which  suggests  evaluation  outcome  of  one  level  is  affected  by  the  preceding  level.  For  example  if  the  criteria  above 
suggest  that  the  intervention  should  be  evaluated  at  the  performance  level,  reaction  and  internalization  evaluation 
outcomes  should  also  be  obtained.  When  a training  intervention  is  implemented,  the  connection  between 
internalization,  as  learning  in  a training  situation,  and  performance  seems  clear  and  logical,  because  if  a person  does 
not  perform,  it  is  possible  that  the  person  lacks  skills  and  knowledge  as  well  as  other  variables.  To  insure  that  the 
person  has  the  skills  and  knowledge  to  perform,  internalization  outcomes  should  be  obtained. 

Selecting  elements 

After  deciding  to  what  extent  an  intervention  is  to  be  evaluated,  using  the  presented  three-dimensional 
matrix  (Figure  1),  practitioners  can  choose  elements  of  the  system.  Using  their  best  judgment,  practitioners  choose 
relevant  elements  from  each  axis  of  the  matrix,  since  each  situation  has  its  unique  features.  For  instance,  while 
conducting  an  evaluation  of  a job  aid  at  individual  level,  it  may  or  may  not  be  helpful  to  consider  external 
economics  and  governmental  factors  for  reaction  outcomes. 

Setting  up  the  evaluation  systems 

In  order  to  determine  the  variables/interventions  that  effect  evaluation  outcomes  for  each  subject  and  their 
interactions,  the  evaluation  systems  involved  need  to  be  made  explicit.  Systems  are  diagrammed  with  elements 
selected  from  the  three-dimension  matrix.  One  important  point  here  is  that  each  evaluation  outcome  should  become 
one  of  the  inputs  of  the  following  level,  because  the  evaluation  outcome  of  preceding  level  is  necessary  for  the 
purpose  of  insuring  that  the  intended  objectives  of  the  intervention  were  achieved.  For  example,  in  a job  aid 
intervention  case,  if  an  employee  did  not  internalize  the  benefits  and  use  of  a job  aid,  trying  to  obtain  only 
performance  evaluation  outcomes  with  this  employee  would  cast  doubt  on  the  success  of  intervention. 

Identifying  data  to  be  collected  and  developing  instruments 

After  determining  evaluation  outcomes,  subject  of  evaluation  and  variables,  next  step  is  to  identify  data  to 
collect  for  each  evaluation  outcome.  Three  dimensions  of  the  model,  subjects,  intervention  and  evaluation  outcome, 
determine  the  criteria  about  the  data  to  be  collected.  From  intervention  to  intervention  and  from  organization  to 
organization  the  nature  of  evaluation  outcome  data  can  change.  The  dimensions  in  the  model  yield  180  (9x5x4) 
different  cells  and  situations,  and  it  would  be  impossible  to  explain  each  of  them  here.  Again  practitioners  need  to 
use  their  best  judgment  to  decide  what  data  to  collect,  but  here  are  some  general  guidelines: 

Define  evaluation  outcomes  operationally : Evaluation  outcomes  need  to  be  defined  operationally.  Although 
each  evaluation  outcome  has  been  defined  previously  at  the  beginning  of  this  section,  practitioners  need  to  answer 
the  question  of  what  constitutes  the  evaluation  outcome  while  considering  the  intervention  and  the  subject  of 
evaluation.  For  example,  when  evaluating  the  internalization  of  a process  improvement  for  the  individual,  a 
practitioner  needs  to  answer  the  question  of  what  indicators  constitute  the  acceptance  and  understanding  of  the  new 
process  for  the  individual.  An  approach  to  this  question  would  be  to  determine  the  awareness  of  the  employee  and 
his/her  knowledge  about  the  process. 

Identify  primary  elements  for  the  evaluation  outcome : Primary  elements  have  direct  and  immediate 
interactions  on  an  intervention  with  subjects  at  an  evaluation  level.  For  example,  the  instructor  and  material  for  a 
training  intervention  can  be  primary  elements  of  the  reaction  outcome  for  an  individual  or  group. 

Identify  secondary  elements  for  the  evaluation  outcome : Secondary  elements  have  indirect  and  non- 
continuous  interactions  on  an  intervention  with  subjects.  Examples  would  be  External  economics  or  the 
governmental  variable  for  training  intervention  at  learning  level. 

After  identifying  the  relevant  data  with  the  help  of  the  three-dimension  model  and  the  evaluation  systems, 
instruments  can  be  developed. 
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Measurement  activities 

Measurement  or  data  collection  activities  are  an  important  part  of  this  model,  they  provide  the  data  to  make 
judgments  about  worth  of  an  intervention  or  attempts  at  further  improvements.  Two  points  can  be  considered.  First, 
several  data  collection  methods  can  be  employed  such  as  surveys,  interviews  and  observations.  Second,  generally, 
evaluating  the  effectiveness  of  an  intervention  process  means  comparing  the  evaluation  outcomes  of  presence  and 
absence  of  the  intervention,  to  this  end,  two  types  of  measurement  activities  can  take  place;  pre  and  post  intervention 
measurement,  or  utilizing  a control  group  (Harris,  1998).  These  two  points  help  increase  the  effectiveness  of  the 
evaluation  process. 


Limitations  and  Further  Application  Recommendations 

The  model  presented  in  this  paper  provides  guidelines  to  evaluate  the  effectiveness  of  performance 
interventions  by  synthesizing  from  two  sets  of  criteria  for  an  ideal  evaluation  model  for  PT.  One  limitation  of  the 
model  is  that  it  only  provides  guidelines  for  selecting  evaluation  methods  and  setting  up  the  forces  for  each 
evaluation  outcome.  It  does  not  provide  specific  methods  or  ready-to-use  procedures.  The  model  relies  on 
practitioners  to  set  up  the  evaluation  systems  for  evaluation.  This  is  one  disadvantage  of  providing  a great  degree  of 
freedom  to  practitioners  and  attempting  to  encompass  a wide  range  of  interventions. 

In  order  to  increase  the  effectiveness  of  this  evaluation  model,  two  recommendations  can  be  considered. 
First,  as  mentioned  earlier,  the  applications  of  Kirkpatrick’s  framework  showed  that  managerial  support  and  creating 
a trust  environment  in  the  organizations  are  crucial  for  obtaining  behavior  or  performance  evaluation  outcomes 
(Harrell,  2001).  In  order  to  successfully  conduct  performance  and  impact  evaluations,  managers  should  be  educated 
about  the  importance  of  evaluation  activities  at  the  performance  and  impact  level  (Riley  et  al.,  2002).  Second, 
institutionalization  of  the  evaluation  process  is  an  important  step  to  increase  effectiveness  of  evaluation  (Smith  & 
Brandenburg,  1991).  Continuous  measurement  activities,  especially  for  performance  and  impact  evaluation 
outcomes,  can  serve  as  indicators  of  the  subjects’  situations  before  and  after  implementation  of  performance 
improvement  interventions.  Measuring  all  the  variables  continuously  in  the  proposed  model  can  indicate  the 
effectiveness  of  an  intervention  when  it  needs  to  be  evaluated. 

Conclusions 

The  focus  of  PT  is  to  improve  individuals’  and  organizations’  performance  through  implementing 
interventions  drawn  from  multiple  disciplines.  After  the  implementation  of  a performance  improvement 
intervention,  it  is  necessary  to  answer  three  basic  questions;  did  the  intended  outcome  occur,  is  the  intervention 
worth  continuing,  and  how  can  the  intervention  be  improved.  This  is  why  an  evaluation  process  should  be 
conducted.  Kirkpatrick’s  four-level  evaluation  framework  is  the  dominant  model  in  the  corporate  world  for 
evaluating  training  and  non-training  interventions.  Although  simplicity  and  practicality  are  the  advantages  of  the 
framework,  its  inadequacy  for  addressing  several  performance  improvement  interventions  and  its  inability  to 
consider  organizations  as  systems  are  signs  of  a need  for  a new  evaluation  model  for  PT. 

The  Dimensions  and  Systems  model  presented  in  this  paper  can  be  an  alternative  for  Kirkpatrick’ s 
framework  to  evaluate  performance  improvement  interventions.  The  Dimensions  and  Systems  is  an  attempt  to 
provide  simple  and  applicable,  yet  adequate,  model  for  practitioners  to  evaluate  effectiveness  of  PT  interventions  in 
their  organizations. 
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Conversation  As  Inquiry:  A Conversation  With  Instructional  Designers 

The  missions,  goals,  and  practices  of  instructional  designers  are  defined  by  organizational  and  cultural 
frames.  What  are  these  frames?  What  images  fill  these  frames  and  who  is  involved  in  creating  them?  If  one  frame 
replaces  another,  how  does  the  focus  change?  Over  the  past  three  years  we  have  been  exploring  this  idea  of  “frames” 
as  a problem  of  practice  for  instructional  designers  who  work  in  institutions  of  higher  education  (HE).  We  believe 
that  instructional  designers  create  meaningful  images  within  frames  that  reflect  constantly  interacting  personal, 
professional,  institutional,  and  social  visions  and  goals.  These  frames  are  described  and  mediated  through  a series  of 
conversations  - with  our  personal  and  professional  histories,  our  discipline,  our  colleagues,  our  clients,  our 
institutional  and  cultural  contexts  and,  most  importantly,  our  own  values.  We  believe  that  there  is  a reciprocal 
relationship  between  design  and  conversation:  conversation  is  design,  and  design  emerges  through  conversation. 
Since  we  live  our  personal/professional  lives  in  a series  of  conversations,  we  believe  that  a series  of  constant 
collaborative  conversations  with  designers  is  a reasonable  approach  to  inquiry.  At  the  same  time  we  know  that 
conversation  is  not  always  deliberate,  nor  do  instructional  designers  always  reflect  deeply  on  conversations  they 
have.  Therefore,  it  was  an  intention  of  our  research  to  have  conversations  with  designers  that  are  deliberate  and  that 
encourage  deep  reflection  about  their  purpose  and  practice,  and  ours ! 

Through  their  work  with  others  instructional  designers  engage  in  a process  of  professional  and  personal 
transformation  that  has  the  potential  to  transform  the  HE  institution,  particularly  through  faculty  client  relationships. 
Borrowing  from  disciplines  as  diverse  as  sociology  and  cognitive  science  we  maintain  that  learning  involves  shared 
thinking  or  understanding,  and  is  most  effective  if  it  is  embedded  in  social  experience  and  situated  in  authentic 
problem-solving  contexts  (cf.  Glaser,  1991;  Jonassen,  Dyer,  Peters,  Robinson,  Harvey,  King,  & Loughner  1997; 
Rogoff,  1990).  Instructional  design,  rarely  played  out  in  social  isolation,  may  be  a form  of  cultural  learning  or 
collaborative  learning  for  all  those  engaged  in  the  interaction.  Exploiting  the  metaphor  of  instructional  design  as 
conversation  has  implications  for  both  personal  and  social  action.  Understanding  this  process  through  conversation 
has  implications  for  both  professional  and  methodological  change  in  our  field. 

In  this  paper  we  explore  four  main  themes — reflexivity,  voice,  strong  subjectivity,  and  power/authority — 
that  are  woven  together  by  the  idea  of  moral  action.  We  look  at  each  theme  twice — once  from  a methodological 
view  or  frame  (i.e.,  conversation  as  inquiry),  and  again  from  a practice-based  frame  (i.e.,  instructional  design  as 
conversation).  This  is  one  of  a series  of  completed  (cf.  Campbell,  Schwier  & Kenny,  2005;  Kenny,  Schwier,  Zhang 
& Campbell,  2005;  Schwier,  Campbell  & Kenny,  2004)  and  developing  papers  that  will  explore  identity,  social 
action  and  community  and  challenge  the  so-called  “grand  narratives”  of  instructional  design. 

The  Research  Study 

The  stories  shared  in  this  paper  were  drawn  from  a three-year  (2002-2005)  study,  funded  by  the  Social 
Sciences  and  Humanities  Research  Council  of  Canada,  involving  six  Canadian  universities  and  smaller  HE 
institutions.  The  participating  institutions  have  an  administrative  and/or  academic  unit  whose  mandate  is  to  support 
faculty  developing  technology-enhanced,  blended,  or  online  learning  environments. 

Methodology 

We  elicited  participation  through  a range  of  strategies  including  personal  email  invitations,  advertisements 
on  listservs  and  blogs,  personal  contacts  at  professional  meetings,  membership  lists  from  professional  associations 
and  visits  to  graduate  classes.  Sources  of  data  include  research  conversations  with  instructional  designers,  email 
correspondence,  and  group  meetings  and/or  focus  groups. 
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The  narrative  study  is  constructed  as  a series  of  collaborative  conversations.  They  refer  to  designers’  lives 
as  learners  and  their  memberships  in  social  and  professional  communities,  their  career  choices,  their  core  values 
about  the  purposes  of  education  and  of  design,  and  their  design  practices,  dilemmas  and  relationships. 

The  Rationale 

We  believe  that  narrative  inquiry  and  the  “storying”  of  experience  are  socially  and  contextually  situated 
interpretive  practices.  We  start  from  personal  stories  because  “personal  knowledge  has  a practical  function. . .as  a 
source  for  deliberation,  intuitive  decisions,  daily  action  and  moral  wisdom”  (Conle,  2000,  p.  51).  That  is,  a deep 
understanding  of  instructional  design  practice  is  most  accessible  to  us  in  the  forms  in  which  designers  actually  do 
design.  Through  a series  of  socially-referenced,  scaffolded  conversations  participants  reveal  how  and  why  they 
design  as  they  do,  and  how  we  can  use  that  understanding  to  prepare  and  support  designers  to  practice  in  the  most 
agentic,  authentic,  and  profound  ways.  For  the  instructional  designers  involved  in  this  study,  telling  stories  of 
practice  requires  a personal  critical,  reflective  engagement  with  the  potential  to  change  or  transform  their  practice. 
Thus  the  methodological  approach  for  the  study  mirrors  a social  constructivist  framework  for  instructional  design 
practice,  which  is  one  of  social  interaction  and  construction  of  meaning  through  conversation. 

The  conversations  we  included  in  this  paper  were  selected  from  the  data  pool  of  49  research  conversations 
(with  20  instructional  designers)  because  these  participants  told  stories  that  very  directly  reflect  the  practice 
dimension  of  conversation.  We  chose  these  conversations  because  they  clearly  reflect  the  idea  of  the  design 
conversation  and  impressed  us  as  effectively  representing  plausible,  authentic,  and  compelling  accounts  that  are  both 
believable  and  invitational. 

Data  Analysis 

Two  researchers  independently  coded  each  transcript  of  the  conversations  using  Atlas  Ti™  software;  the 
third  completed  a broader  thematic  analysis.  That  is,  two  researchers  independently  used  a micro  coding  approach 
while  one  examined  the  transcripts  with  a wider  lens.  As  themes  emerged,  they  were  negotiated  with  the  entire 
research  team  and,  where  possible,  shared  with  the  participants.  The  themes  were  then  used  to  construct  networks  of 
meaning.  This  reflexive  process  is  intended  to  further  engage  participants  in  identifying  issues  related  to 
instructional  design  by  bringing  the  personal  and  community  problems  of  practice  into  self-awareness.  In  this  way, 
narrative  inquiry  involves  the  “politics  of  identity  construction  and  ongoing  identity  maintenance”,  where  the  lived 
experiences  of  instructional  designers  can  “be  used  as  the  sites  wherein  and  whereby  we  interrogate  the  social  world 
theoretically  and  critically”  (Goodson,  1995,  4). 

Conversation  As  Inquiry:  An  Alternate  Frame 

It  is  generally  accepted  that  ID  practice  was  originally  based  on  the  behaviorist  learning  theories  of  Skinner 
and  Thorndike,  among  others  (Saettler,  1992),  and  that  instructional  design  was  based  on  the  empiric  assumption 
that  behavior  is  predictable,  and  that  educational  design  can  occur  in  isolation  from  the  contexts  in  which  learning 
will  take  place  (Koper,  2000,  8).  As  a result,  the  language  of  design  has  reflected  a systematic  approach  based  on 
social  engineering  and  reflecting  the  values  of  efficiency  and  effectiveness  (cf.  Merrill  & ID2  Research  Group,  1996; 
Dick,  Carey  & Carey,  2005).  These  are  supposedly  value-free  ways  of  shaping  and  representing  knowledge  based  on 
the  assumption  that  educational  technologies  and  environments  are  neutral  and  democratic  and  that  knowledge  can 
be  codified  and  presented  in  templates  or  blueprints  that  describe  reality.  Designers,  programmers,  and  media 
developers  emerging  from  this  “scientific”  field  have  learned  models  that  value  objective,  rational,  instrumental,  and 
empirical  approaches  (e.g.,  Braden,  1996).  Critical  theorists  have  described  the  products  and  environments  they 
produce  and  deliver  as  too  often  prescriptive,  restrictive,  and  reductionist,  due  in  no  small  way  to  the  culture  they 
have  acquired  by  their  training  (c.f.  Garrison,  1993;  Vrasidas,  2001). 

Research  about  instructional  design  has,  until  very  recently,  also  reflected  an  objective,  rational  approach. 
The  behavioral  science-based  approach,  in  particular,  began  to  have  an  increasing  impact  on  the  field  in  the  1970’s 
as  researchers  sought  a design  science  “as  developed  by  experts  in  the  areas  of  psychology,  anthropology,  sociology, 
learning,  group  processes... communications,  cybernetics,  perception,  psychometrics,  human  factor  engineering, 
branches  of  economics  such  as  personnel  utilization,  and  computerized  systems”  (Saettler,  1990,  12).  The  cognitive 
approach  to  inquiry  emerged  from  the  behavioral  science  approach  and  endeavored  to  understand  a learner’s  internal 
cognitive  processes,  such  as  thinking,  organizing  and  remembering,  and  activating  the  appropriate  learning 
strategies  during  the  instructional  process.  This  approach  considered  media  attributes,  learner  strategy 
implementation,  and  task  analysis  (Saettler,  1990).  The  search  for  a design  science  engendered  hundreds  of  studies 
of  experimental  design  examining  learning  conditions  and  outcomes,  many  of  which  revealed  “no  significant 
difference”  (c.f.  Hannafin  & McDermott,  1995;  Russell,  1999).  In  all  these  approaches  the  instructional  designer’s 
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scope  was  restricted  to  systematically  choosing  and  applying  the  design  model  likely  to  yield  the  most  effective 
instruction.  The  design-science  vision  has  underpinned  its  dominant  cognitive  science  culture  for  many  decades  and 
eliminates  the  messy  process  of  problem  solving  where  non- systematic  human  thoughts  and  emotions  are  involved 
(c.f.  Willis,  1998). 

But,  there  is  a cultural  shift  in  ID,  framed  by  social  constructivists  for  whom  is  constructed  in  communities 
of  practice  through  social  interactions.  Cobb  (1996)  argues  that  knowledge  is  not  held  objectively,  but  is  unique, 
wholly  subjective,  and  passed  on  by  establishing  common  ground  between  the  knower  and  the  learner.  That  is, 
“instruction  and  instructional  technology  are  human  inventions  that  spring  from  human  values  and  human  designs. 
They  are  value-saturated  and  operate  in  the  social  world. . .social  inventions. . . are  never  value-free  or  value-neutral. 
They  resonate  with  the  values  of  their  human  creators,  who  themselves  are  situated  in  a particular  culture  in  a 
specific  time  and  place”  (Johnsen  & Taylor,  1995,  94).  After  Connelly  and  Clandinin  (1990)  we  see  instructional 
design  practice  in  an  alternative  way:  knowledge  is  embodied  within  individuals  in  a relationship,  for  example 
faculty  clients/collaborators  and  instructional  designers,  who  together  interpret  their  experiences  through  personally 
and  socially  constructed  visual  and  verbal  forms,  whether  that  be  language,  pictures,  or  actions. 

Since  design  decisions  are  often  experiential,  intuitive,  aesthetic,  and  phenomenological,  related 
instructional  design  research  should  reflect  critical  paradigms  that  “provide  a mode  of  inquiry  which  can  provide 
insight  and  information  which  goes  beyond  the  possibilities  of  scientific  inquiry. . .(into)  the  realm  of  art”  (Hlynka  & 
Belland,  1991,  9).  Here  we  make  a case  for  narrative  as  a form  of  critical  inquiry  in  instructional  design  practice. 
The  unstructured  interview  design,  or  collaborative  conversations,  captures  the  participants’  own  constructions  of 
experience,  families  and  social  cultures:  seminal  personal  and  professional  encounters;  moral  and  ethical  beliefs  and 
dilemmas;  development  and  understanding  of  their  work  as  instructional  designers,  and  how  their  knowledge  is 
embodied  in  their  relational  practice  with  faculty.  We  explore  four  main  ideas  as  methodological  issues:  reflexivity, 
voice,  strong  subjectivity,  and  power/authority. 

Theme  1:  Reflexivity 

These  designers  told  us  that  they  design  by  asking  faculty  to  tell  stories  about  their  teaching,  and  reflecting 
these  stories  back  through  the  learning  design.  Storytelling  recounts  personal  action  and  reflects  learning  for  both 
faculty  and  designer.  In  a design  conversation,  both  designer  and  faculty  are  critically  aware  of  how  each  other’ s 
role  affects  the  telling  and  challenges  the  other’s  construction  of  the  story.  What  emerges  is  a socially  transformative 
story  of  the  design  process  (Campbell,  1994).  The  narrative  conversation  is  a fundamental  activity  of  mind, 
involving  an  intentional  reflective  activity  (Lyons  & Kubler  LaBoskey,  2002).  As  we  talk  to  the  designers  about 
their  past  projects  the  temporal  distance  requires  a cognitive  and  emotional  reconstruction  of  the  experience.  In  its 
collaborative  form  the  conversation  becomes  an  opportunity  for  growth  for  both  designer,  who  has  new  knowledge 
to  use,  and  us  (who  are  also  instructional  designers),  whose  new  appreciation  for  the  design  process  is  reflected  in 
both  refined  methods/better  questions  and  enhanced  personal  design  practice. 

Theme  2:  Voice 

As  a conversation-based  practice,  instructional  design  is  pluri vocal.  In  the  conversation,  clients  and 
designers  and  researchers  share  space  and  moral  authority  in  the  acts  of  listening,  taking  turns,  attenuating;  acting  in 
each  other’s  stories.  The  story  is  “a  statement  of  belief,  of  morality,  it  speaks  about  values”  (Goodson,  1995,  12). 
Storying  experience  establishes  complex  interrelationships,  is  social  and  political  and  assumes  equity.  In  the  end  no 
one  owns  the  story,  because  the  research  story  would  not  exist  without  its  telling  and  its  hearing;  its  resonance  and 
dissonance  with  the  listener’s  experiences  (Gudmundsdottir,  1998).  As  researchers,  we  can  interrogate  design 
practice  through  active  social  participation  in  the  conversation,  because  storying  the  design  experience  requires 
conscious  knowledge  and  understanding  of  action  (Lyons  & Kubler  LaBoskey,  2002). 

Accountability  to  the  storyteller  and  the  story  becomes  both  an  ethical  and  a methodological,  or 
interpretive,  issue  in  this  research.  Even  though  we  intend  the  conversations  to  be  collaborative  and  equitable,  by 
recounting  the  conversation  we  have  also  co-opted  it.  For  example,  in  a conversation  there  are  silences,  facial 
expressions,  vocalizations;  certain  stories  become  unavailable.  If  we  listen  to  the  original  tape  during  the  transcript 
analysis  we  may  be  aware  of  these  “meaning  units”  but  may  choose  not  to  include  them.  If  we  were  not  present 
during  the  conversation  and  are  working  with  transcripts  only,  we  might  not  realize  that  an  extended  silence 
occurred  because  the  designer  might  have  been  struggling  to  gain  control  of  his  emotions.  In  one  transcript,  our 
transcriber  inserted  comments  like  “laughs  nervously”  and  “sighs  heavily”,  which  was  only  her  interpretation  of  the 
emotional  context  of  the  conversation.  Another  transcriber  did  not  insert  any  affective  or  descriptive  markers  in  the 
transcripts.  Listening  to  the  tape  from  an  earlier  conversation  one  researcher  realized  that  she  was  talking  more  than 
the  designer,  an  indication  of  unequal  voice.  However,  “mutuality”  is  another  way  to  interpret  this  interaction:  the 


114 


stories  were  so  resonant  that  the  conversation  reflected  deep  connections  in  parallel  finished  thoughts  and  shared 
cultural  memories. 

Theme  3:  Strong  Subjectivity 

As  instructional  design  is  a social  project,  so  too  is  understanding  the  process,  an  act  with  moral  and 
political  undertones.  Moral  in  that  the  design  conversation  reflects  values  and  beliefs  that  guide  practice,  and 
political  in  that  practice  at  least  implies  the  possibility  for  social  action  in  the  institution.  As  designer-researchers  we 
are  agents  of  change,  because  we  can  retell  design  stories  in  ways  that  challenge  the  dominant  research  culture  of 
instructional  technology,  or  dominant  practices  in  HE.  Our  shared  experiences  with  designers,  rather  than  our 
differences,  makes  us  activists,  because  we  have  chosen  issues  close  to  our  own  hearts  and  feel  responsible  to 
represent  this  social  process.  In  essence  this  is  why  the  research  conversation,  like  the  design  conversation,  is  a 
moral  rather  than  a technical  act  (Herda,  1999). 

As  instructional  technology  has  been  concerned  by  a reproducible  product  in  practice  alternative  research 
frames  have  been  held  up  to  the  “quality  measures”  common  to  the  positivist  requirement  for  “truth  claims”,  that  is, 
reliability,  validity,  generalizability;  the  rigor  associated  with  an  “objective”  research  design.  But  narrative  inquiry, 
in  which  conversation  is  both  method  and  data,  is  more  critically  concerned  with  ethical  conduct,  voice,  reflexivity, 
and  resonance.  The  researcher  brings  his  or  her  “cultural  self’  as  a set  of  resources  for  the  relational  work  of 
narrative.  Utilizing  oneself  as  an  interpretive  resource  is  critical  to  understanding  how  the  research  process  is 
unfolding  and  how  meaning  is  made  in  conversation.  In  other  words,  identifying  the  interviewer  as  a designer- 
researcher  permits  us  to  use  the  commonly  understood  language  of  design.  A shared  language  enables  us  to  “create 
and  acknowledge  meaning  as  we  engage  in  discourse  and  fulfill  social  obligations. . .(that)  are  characterized  as  moral 
activities”  (Herda,  1999,  24).  We  become  part  of  the  past  and  the  future  as  “temporal  and  social,  cultural  horizons 
are  set  and  reset”  (Connelly  & Clandinin,  1990,  5)  through  the  conversation;  our  stories  are  merged  with  those  of 
our  participants  and  emerge  as  scripts  for  further  practice  or  action.  The  quality  measure  here  is  the  moral  obligation 
we  share  as  designer-researchers  with  designers  to  “tell  it  the  way  we  now  understand  it”,  rather  than  as  “the  way  it 
was”.  This  strong  subjectivity  requires  that  we  investigate  rather  than  deny  our  relationship  with  our  conversation 
partners  and  to  explicitly,  consciously  and  critically  examine  and  respond  to  the  interpersonal  dynamics.  The 
relationship  between  us  pervades  every  aspect  of  the  process,  determining  the  quality  and  quantity  of  the 
information  gathered  (Cole,  1991).  A reciprocal  relationship  is  enhanced  by  the  collaborative  sharing  of  personal 
experience  as  a component  of  trust,  rather  than  an  effort  to  maintain  an  objective  distance  during  conversations 
about  experiences  we  all  share. 

Theme  4:  Power  and  authority 

Empowerment  is  a goal  of  conversation-as-inquiry,  which  is  political  in  the  same  way  as  this  project, 
having  a different  purpose  from  historical  and  traditional  forms  of  educational  technology  research,  is  political.  The 
historical  purpose  of  the  latter  has  been  to  gain  knowledge  for  the  power  to  define  “evidence-based”  instructional 
design  models  and  control  over  the  ways  that  designers  are  supposed  to  practice.  The  stories  we  are  hearing  and 
retelling  are  potentially  subversive  and  disruptive,  because  they  encourage  us  to  see  the  design  world  and  understand 
our  own  design  practice  in  new  and  different  ways.  They  become  counter-narratives  of  instructional  design. 
(Elbaz-Luwisch,  1997;  Kanno,  1997;  McEwan,  1997). 

The  relationships  that  designers  build  with  their  clients  often  reflect  their  “ethic  of  care”  for  vulnerable 
faculty  dealing  with  the  cultural  expectations  of  HE  institutions  (Noddings,  1991;  Schon,  1987),  and  at  the  same 
time  with  the  issue  of  class  membership  in  that  same  hierarchical  culture.  We  have  heard  stories  of  struggle  for 
identity  and  credibility  that  resonate  with  our  own  experiences  as  designers,  and  as  qualitative  researchers  in  this 
field  (Schwier,  Campbell  & Kenny,  2004).  However,  it  came  as  a surprise  that  our  designers  sometimes  felt  the 
same  inequity  in  power  and  authority  with  us.  This  raises  questions  about  power  and  authority  within  a research 
relationship  which  is  defined  as  collegial  by  the  researcher,  who  is  also  a designer,  and  the  participant,  who  feels 
uneasy  about  fully  disclosing  conflicts,  mistakes;  times  of  doubt  in  her  practice.  This  sort  of  dilemma  highlights  the 
differences  in  epistemology  and  purpose  between  objective  (traditional  experimental  design)  and  emancipatory 
(narrative)  forms  of  research.  Shotter  and  Logan  (1988)  suggest  that  the  former  leads  to  a “decontexted  kind  of 
theoretical  knowledge. . .expressed  in  a hierarchically  arranged,  closed  system  of  binary  oppositions. . .concerned 
with. . .a  unity  of  vision  and  thought. . .everything  in  its  proper  place  and  all  conflict  eradicated,  once  and  for  all” 

(75).  Emancipatory  forms  contextualized  by  relationship,  on  the  other  hand,  make  space  for  a “plurality  of 
otherwise  conflicting  voices”  (75). 

It  doesn’t  surprise  us  that  designers  struggle  with  identity  and  agency  in  a culture  that  privileges  individual, 
monological  knowledge  held  in  an  authoritative  closed  hierarchy  (HE),  and  in  a research  culture  that  is  dominated 
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by  the  “discovery  of  yet  more  techniques  of  control,  use,  appropriation. . (Shotter  & Logan,  1988,  75).  The  current 
project  positions  us  as  members  of  the  design  and  the  research  communities  with  a particular  vision:  as  designer- 
researchers  we  can  challenge  the  power  dynamics  in  educational  technology  research,  we  can  democratize  the 
structures  that  specify  the  sphere  of  the  designers’  professional  action  by  clearing  space  for  their  authentic  stories. 
Practice  is  contingent  on  social  context  and  what  we  learn  about  practice  from  these  conversations  is  likewise  of 
practical  and  contingent  use  in  professional  development,  graduate  study,  and  research  contexts.  Capturing  the  idea 
of  designer-researcher  agency  Herda  (1999)  proposes  that  by  grounding  the  notion  of  action  in  moral  decisions,  and 
changing  our  idea  and  understanding  of  professional  identity  “we  change  our  actions  as  researchers  within  the 
broader  professional  community  so  that  our  findings  may  take  on  a significance  in  our  own  lives  and  in  the  lives  of 
our  participants”  (91). 


Instructional  Design  As  Conversation 

In  the  second  part  of  this  paper  we  explore  critical  questions  about  the  conversation-based,  social  practice 
world  of  the  instructional  designer  through  the  four  integrating  ideas  of  reflexivity,  voice,  strong  subjectivity,  and 
authority.  These  ideas  are  articulated  through  the  pseudonymous  voices  of  three  designers,  Steve,  Yan,  and  Maria. 

Steve  and  Wilhelm:  Conceptual  Constraints  and  Reciprocity 

Steve  came  to  as  an  intern  from  a graduate  program  that  was  significantly  project-based;  the  university  was 
ideally  situated  to  attract  clients  from  the  corporate  sector.  Graduate  students  learned  the  “evidence-based”  models 
(Steve  talks  a lot  about  ADDIE)  and  became  skilled  at  experimental  research  design.  Steve  was  attracted  to  the 
internship  placement  because  the  hosting  development  centre  at  another  university  had  just  been  established  as  a 
centrally  supported  unit.  He  was  pleased  to  be  able  to  contribute  to  “building  processes  internally. . .so  a lot  of  the 
design  work  was  actually  experiential.”  The  centre’s  approach  of  working  with  faculty  in  a professional 
development  model  encouraged  him  to  work  intensively  with  faculty  based  on  their  personal  connection  with  each 
other.  That  is,  the  centre  emphasized  faculty  learning  over  more  technical,  course  production  goals.  His  design 
practice  involved  a long  phase  of  informal  conversations  followed  by  another  phase  of  formal,  team-oriented  design 
meetings.  Steve  felt  that  faculty  learning  happened  in  the  first  phase,  where  in  the  second  phase  design  decisions 
were  consolidated  and  implemented  by  a more  technical/production  group. 

Steve  talked  retrospectively  about  one  project  in  particular  that  had  the  most  influence  on  his  growth  as  a 
designer.  Wilhelm,  the  faculty  client,  was  an  internationally  renowned  teacher/philosopher,  who  was  initially  very 
skeptical  about  the  support  that  a virtual  environment  could  provide.  A pragmatist,  Steve  had  his  own  doubts  about 
the  nature  of  hermeneutic  inquiry,  admitting  that  he  is  “more  empirically  minded  so  a lot  of  what  he  would  talk 
about  I couldn’t  buy.”  However,  these  tensions  helped  the  design  conversation  evolve  as  each  struggled  to 
understand  the  knowledge  base  and  values  of  the  other. 

As  Wilhelm  described  his  instructional  approach,  Steve  visualized  “lessons  that  would  just  sort  of 
unfold. . .they  are  not  designed,  they  are  not  planned,  they  unfold  and  that  is  part  of  that  whole  (hermeneutic) 
philosophy. . .allowing  that  to  happen.”  Accepting  that  this  was  a viable  model  challenged  Steve’s  beliefs  about  a 
designed  learning  environment.  Wilhelm’s  teaching  was  conversation-based,  reflecting  his  deep  curiosity  about  how 
individuals  live  in  and  understand  their  world.  Steve  remembers  that  “there  was  always  this  conversation  about 
‘Well,  how  do  you  interact  with  the  students?’  and  he  would  describe  things  that  he  did  in  the  classroom.” 

Wilhelm’s  instructional  goals  were  embedded  in  his  experiences  of  and  beliefs  about  living  actively  in  the  social 
world,  and  his  responsibility  as  a teacher  to  represent  that  kind  of  relationship  with  life  phenomena. 

We  were  also  talking  about  how  we  could... really  to  allow  it  so  it  wouldn't  need  to  be  managed  (by 
Wilhelm)... we  came  up  with  a number  of  activities  including  things  where  students  would... be  given 
pictures  and  then  they  would  quickly...  respond  to  the  pictures  with  their  feelings  or  with  their  thoughts  or 
whatever  the  picture  evoked...  they  would  go  through  a number  of  these  and  they  would  be  randomly  pulled 
out  of  the  database  and  then  they  would  go  through  them  again  but  with  other  people's  comments  attached 
and  then  they  would  make  comments  and  then  ( Wilhelm ) would  use  these... to  talk  about. . .different  things 
about  the  experience... 

The  resulting  design,  an  interactive  concept  map,  is  one  outcome  of  a process  in  which  Steve  struggled  to 
understand  a way  of  looking  at  the  world  so  that  he  could  help  Wilhelm  faithfully  represent  online  the  dialogic 
nature  of  phenomenology.  But  while  at  times  the  design  conversation  was  “over  my  head”  there  was  also  a “deeply 
cultural  reason”  for  liking  and  personally  committing  to  Wilhelm. 

my  background  is  German... (his)  accent  always  reminds  me  of  members  of  my  family...  and  one  of  my 
favorite  high  school  teachers  was  actually  a student  of  (his).  He  did  his  Masters  with  Wilhelm....  I 
remember...  this  teacher  talked  about  Wilhelm.. .and  then  his  son  was  my  best  friend  in  high  school... 
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( Wilhelm ) always  reminds  me  of  that  teacher... because  my  teacher  was  like  Wilhelm...  There's  this  certain 
way  of  looking  at  things. 

We  picked  up  on  the  notion  that  a shared  cultural  heritage  opened  up  a space  for  Steve  and  Wilhelm  to 
work  in  harmony,  although  Wilhelm’s  project  caused  some  cognitive  dissonance  for  both.  Steve  responded  that  his 
doubts  actually  allowed  him  to  ask  the  questions  that  encouraged  Wilhelm  to  interrogate  his  own  assumptions, 
and. . .’’start  breaking  things  down. . .pieces  and  examining  assumptions  and  helping  me  to  help  them  understand 
where  the  learning  problems  are  gonna  occur.” 

We  see  through  this  exchange  that  the  agency  relationship  is  reflexive;  both  participants  influence  the 
cognitive  processes  of  the  other  and  as  a result  both  change.  Certainly  the  amount  of  reciprocity  between  a designer 
and  client  is  defined  by  the  relationship  and  the  institutional  context  within  which  their  relationship  operates.  For 
example,  an  institution  that  emphasizes  a hierarchical  relationship  between  faculty  and  designers  introduces  power 
and  authority  to  the  conversations.  Reflexivity  can  introduce  dissonance  to  the  conversations,  and  the  resolution  of 
that  dissonance  can  result  in  designs  that  are  marked  by  nuance  and  depth.  The  relationship  is  moral  because  it  leads 
to  positive  action.  The  design  does  not  even  have  to  come  to  fruition;  it’s  the  process,  the  conversation,  and  the 
relationship  that  is  important.  Steve  acknowledges  his  personal  and  institutional  agency  in  faculty  clients’  changed 
perspectives. 

People  (that)  would  come  back  afterwards  and  say,  “I'm  teaching  differently  now.  Because  of  our 
interaction  I’m  teaching  differently  " and  that  would  be  like  an  epiphany,  right?  Even  Dr.  Stone... (whose) 
project  never  got  off  the  ground,  he  would  come  back  and  we  would  talk  about  other  things... we  would  talk 
about  the  problems  he  was  having,  we  would  talk  about  ways  that  he  could  address  them,  and  afterwards 
him  saying,  you  know,  (Tm  changing  the  way  that  I'm  looking  at  these  instructional  problems  for  my 
classroom.  " 

Yan  and  Design  as  Activism 

Yan  is  typical  of  many  in  this  study  who  “cut  their  design  teeth”  in  settings  other  than  universities,  and 
before  obtaining  formal  graduate  education  in  the  field.  We  have  observed  that  these  designers  tend  to  bring  both  a 
pragmatic  and  a political  sensibility  to  their  practice  at  universities.  For  example,  Yan  and  Maria,  the  designers  in 
the  next  two  stories,  understand  the  practice  of  instructional  design  as  a way  to  embody  their  goals  of  accessibility 
and  inclusion.  Yan  refers  to  her  background  as  lower  middle-class,  giving  her  an  interpretive  lens  in  the  80’ s and 
90’ s as  she  worked  in  various  community  service  settings  with  “the  disenfranchised”. 

It  seems  to  me  that...  there's  kind  of  a political  landscape  that  has  allowed... people  who  are 
disenfranchised  or  just  disadvantaged  in  some  way,  to  be  pushed  even  further  down  in  terms  of  their 
own... opportunities....  That  has  been... a consciousness  that  has  evolved  for  me  in  the  past  twenty  years. 
Aware  of  the  lack  of  social  engagement  of  some  faculty,  Yan  talks  about  teaching  and  design  as  political 
acts  whose  purpose  is  to  improve  the  kinds  of  learning  opportunities  that  will  lead  to  social  justice.  Soon  after 
obtaining  an  undergraduate  education  degree  she  took  a full-time  position  as  an  ESL  instructor  in  a community 
college  with  a large  student  population  of  political  refugees.  Her  colleagues  “had  a very  radical  orientation  to 
education,  were  quite  active  in  a number  of  political  movements, 

awakening  my  awareness  to  larger  issues  because  when  you  talk  about  teaching  in  that  context,  you  're  not 
just  talking  about  content;  you're  talking  about  a wider  role,  like  what  are  issues  of  social  justice?... it's 
quite  interesting  for  me  to  come  (to  this  university)  and  work  with  professors  sometimes  who  are  very 
knowledgeable  in  their  area,  but... it's  hard  for  them  to  convey  to  their  students  what  the  significance  of 
that  content  is  within  the  wider  context  of  the  world.  So  that's  an  interesting  situation  for  me... not  as  a 
result  of  my  formal  education;  it's  more  my  informal  education  through  working  with  people  who  were 
very,  very  committed  educators... who  really  walked  the  talk  of  what  they  believed...  who  were  very  much 
into  people  like  Freire... 

As  a result  of  this  powerful  socialization  to  critical  frameworks  for  education,  Yan  decided  to  pursue  an 
interdisciplinary  Masters  degree  from  both  adult  education  and  educational  technology,  hoping  that  learning 
technologies  would  broaden  access  for  the  groups  to  whom  she  had  been  committed  for  two  decades.  Instead,  in  her 
program  she  found  a “micro-focus”  on  technical  skills.  When  we  talked  about  ways  to  have  the  sociopolitical  impact 
engendered  by  a more  critical  focus  for  instructional  design  she  pointed  to  writing  for  the  professional  community — 
she  was  working  on  a chapter  for  a book  on  technological  and  information  literacy.  Interested  in  equality  (as). . .a 
value  that  I bring  to  the  table”  and  seeing  “nothing  in  there  in  terms  of  marginalized  groups”  she  decided  to 
contribute  a chapter  on  digital  divide  issues. 

Yan’s  practice  as  a designer  is  framed  by  a moral  purpose,  which  she  often  disguises  when  working  with 
faculty  with  more  instrumental  goals.  She  describes  one  project  in  which,  by  having  a series  of  conversations  about 
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her  own  experiences  as  a teacher  of  political  refugees,  she  convinced  a client  to  develop  case  studies  framed  by  a 
relevant  social  issue. 

In  a couple  of  different  case  studies... I tried  to  make  (the  family)  lower  middle  class  in  terms  of  how  they 
talked...  one  of  the  characters  decided  to  become  a vegetarian  and  she  announced  at  the  dinner  table  that 
she  was  an  ovo-lacto-vegetarian  and  her  father  said,  ‘ What  the  hell  is  that?’  And  the...  instructor  said, 

‘ That’s  really  funny.  That’s  exactly  what  my  father  would  have  said.  ’....  I kind  of  wanted  to  set  the  stage— 
here’s  a family  that  is  struggling  financially...  the  father  loses  a job....  The  daughter  is  in  university...  and  in 
the  very  last  story  the  daughter  is  visiting  the  food  bank  at  the  student  union.  So  we  have  a whole  case 
study  in  terms  of  student  hunger.  We  wanted  to  make  the  case  studies  relevant  to  student  life  and  the  fact 
that  there  are  probably  people  on  campus  who  don’t  have  enough  money  for  a balanced  diet. 

Yan  identified  this  project  as  one  where  she  felt  her  own  social  values  were  aligned  with  the  goals  of  the  faculty  in 
nutrition  “because  there’s  a social  science  element  to  (their  practice). . .they’re  trying  to  include  things  in  their 
introductory  class—  like  aboriginal  health. ...  social  issues  are  part  of  that  mixture. ...  The  people  who  work  there 
reflect  that.”  Because  her  sociopolitical  values  harmonized  with  those  of  these  faculty  clients  she  felt  more  effective, 
her  voice  was  heard  and  incorporated  in  the  design;  her  instructional  design  practice  embodied  societal  agency. 

However,  while  at  times  Yan  is  able  to  work  in  moral,  relational  ways,  when  her  clients  focus  on  content 
outside  of  the  context  of  significant  cultural  political  issues  she  wonders  whether  she  has  any  impact  on  the  socially 
detached  culture  of  the  university.  She  looks  back  at  her  early  career  “dealing  with  people  who. . .saw  half  their 
family  get  killed,  and  now  they’re  in  a new  country,  and  they  have  to  learn  the  language  and. . .get  a job. . .”  She  has 
identified  other  ways  to  measure  success  that  reflect  her  global  social  conscience,  “so  I have  to  measure  success  in  a 
different  way. . . the  instructor  has  acknowledged  that  there  are  more  people  in  the  class  other  than  the  keeners. . .. 
And  I really  feel  the  few  things... like  the  teachable  moments  (with  faculty). . . have  been  very  worthwhile.” 

Maria  and  Intellectual  Wrestling 

Maria  feels  that  her  instructional  design  career  reflects  her  life,  as  part  of  being  “constantly  in  process.”  She 
credits  relationships  with  colleagues  and  faculty  with  being  “extremely  instrumental”  in  helping  her  define  her  goals 
“flow(ing)  towards  what  I think  now  is  more  authentically  what  I need  to  be  doing.”  Maria’s  design  approach  is  one 
of  strong  subjectivity,  referenced  by  her  memories  of  learning,  and  of  teaching  in  a college,  seeing  explicit 
connections  between  her  experiences,  values  and  practice. 

Maria  told  the  story  of  a development  project  for  a theology  course.  The  instructor  was  untenured,  newly 
hired  and  still  working  on  his  dissertation.  Maria  had  met  him  and  connected  on  a deep  level  with  him  as  she 
respected  that  “religion. . .really  gets  to  the  heart  of  who  a person  is  and  how  they  interpret  the  world”  and  she  was 
interested  in  exploring  how  his  spiritual  lens  would  frame  the  design  process.  She  found  that  he  did  not  have  a lot  of 
experience  in  the  classroom  and  that  he  initially  characterized  the  Internet  as  a “tool  of  war  and  social  aggression.” 
His  own  experience  in  the  teaching  process  had  been  very  lecture -oriented  and  his  vocation  as  a priest  is 
very  intellectually-oriented...  in  the  first  few  meetings  he  came  across  as...  quite  skeptical  about  whether  or 
not  a course  of  this  nature  could  be  taught. 

Maria  planned  to  “set  up  some  cognitive  dissonance”  through  a series  of  conversations.  They  met  many 
times,  informally,  to  talk  about  the  meaning  of  worship  and  the  goal  of  religion  in  the  schools.  She  attended  a Mass. 
She  describes  her  agentic  role  in  course  development  as  facilitative,  respecting  the  relationship  and  at  the  same  time 
requiring  “a  sort  of  intellectual  wrestling. . ..  posing  challenging  questions  about  what  it  is  they  are  doing,  why  they 
are  doing  it;  why  they  think  that  might  work. . . perhaps  creating  enough  discomfort  about  that,  (so)  that  change  can 
happen.”  Although  the  instructor  was  initially  resistant  to  the  idea  of  technology,  and  as  a priest  in  a theological 
college  was  cloaked  in  both  institutional  and  moral  authority,  she  didn’t  feel  resistance  from  him  on  a personal  level. 
She  noted  that  “after  several  months. . .he  was  starting  to  integrate  things  that  we  had  talked  about  even  in  his  face- 
to-face  version  of  his  class  and  that  he  was  thinking  about  and  wrestling  with  the  way  he  was  teaching,  what  that 
meant...” 

Maria  explicitly  resists  the  expectation  for  an  optimal,  reproducible,  impersonal  course  design.  If  a client’s 
“absolute  insistence”  on  a particular  approach  is  ill  advised,  she  will  still  try  to  “somehow  honor  that  piece  and  come 
up  with  creative  ways  of  making  it  happen  that  will  be  useful  to  learning.”  Her  ethic  of  care  for  the  person  involved 
in  the  process  “comes  back  to  my  original  feeling  that  this  course  has  to  be  a reflection  of  the  person  that  is  teaching 
and  has  to  be  part  of  their  personality,  they  have  to  buy  into  it.”  Maria  is  purposeful  in  a reflexive  design 
conversation  that  values  a more  reciprocal,  relational  process,  one  in  which  both  participants  experience  profound 
growth.  In  this  way  she  reflects  back  a passion  through  a sort  of  affective  resonance. 

...  he  said... that  what  he  found  in  me  was  a great  conversationalist  and  that  we  were  able  to  talk  through 
lots  of  things....  he  actually  said  he  always  felt  very  empowered  to  do  this  and  that....  for  both  of  us  the 
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discussions  were  always  enjoyable , were  always  an  exchange... whatever  the  content  (he)  was  trying  to 
address  we  would  end  up  in  conversation  about  that  ...he  was  very  forthcoming. . .there  was  a...  kind  of 
mutual  interest  in  terms  of  him  being  interested  in  what  I was  able  to  offer  as  an  instructional  designer  and 
my  interest  in  his  content  and  so  there  was  always  an  exchange. 

During  the  research  conversation  with  Maria,  we  were  invited  to  the  course  launch  in  the  theological 
college,  which  was  framed  by  a formal  ceremony  of  worship  and  celebration.  The  instructor  talked  very  movingly 
about  his  transformed  teaching  practices  and  his  acceptance  of  the  Internet  as  “an  enabler  of  peace  and  community”. 

Final  Thoughts 

At  the  beginning  of  this  paper  we  framed  instructional  design  as  an  active  practice  based  on  community, 
practical  reasoning,  personal  perspective,  and  semantic  innovation—  involving  memory  and  leading  to  action. 
Exploiting  the  metaphor  of  instructional  design  as  conversation  has  implications  for  both  personal  and  social  action. 
Understanding  this  process  through  conversation  has  implications  for  both  professional  and  methodological  change 
in  our  field,  because  the  narrative  conversation  has  moral,  emotional,  aesthetic  and  intellectual  dimensions,  and 
design  is  all  of  these. 

More  than  a decade  ago,  Murphy  and  Taylor  (1993)  described  their  experiences,  as  faculty  members,  in  a 
course  development  project.  Their  aim  “in  telling  the  tale  is  to  show  that  the  development  of  learning  materials  does 
not  take  place  according  to  the  kinds  of  models  that  one  typically  comes  across  in  most  instructional  design 
textbooks.”  Arguing  that  such  models  “restrict  rather  than  foster  the  kinds  of  creative  processes”  necessary  for 
effective  design,  they  referred  to  the  emotional  and  intellectual  complexity  of  design  by  describing  the  process  of 
“confusion,  conflict,  ambiguity,  and  uncertainty” (45). 

The  reality  is  that  this  “messiness”  (c.f.  Schon,  1987)  should  not  be  seen  as  a problem  to  be  overcome,  but 
as  a stimulating  and  creative  environment  in  which  relationships,  rather  than  content,  are  at  the  center  of  the  action. 
Connelly  and  Clandinin  (1990)  refer  to  the  central  task  of  narrative  as  representing  “people  (that)  are  both  living 
their  stories  in  an  ongoing  and  experiential  text  and  telling  their  stories  in  words  as  they  reflect  upon  life  and  explain 
themselves  to  others.”  For  us  this  is  both  the  central  task  of  instructional  design  and  the  power  of  narrative  to  reflect 
that.  The  stories  we’ve  included  here  demonstrate  the  kinds  of  things  we  can  learn  when  we  use  conversation  as  a 
lens  into  our  practice.  An  instructional  design  process  in  which  faculty  and  designer  work  toward  the  “imagined 
future”  of  the  designed  learning  environment  is  a “matter  of  growth”  that  “involves  retelling  stories  and  attempts  at 
reliving  stories”  (4). 

Narrative  implies  relationships  with  moral  dimensions — trust,  reciprocity,  reflexivity,  plurivocality— that 
leads  to  critical  reflection  and  has  profound  practice  implications.  Instructional  designers  in  higher  education 
workplaces  use  narrative  to  assist  faculty  clients  tell  the  instructional  story;  they  have  thousands  of  stories  to  share. 
All  we  have  to  do  is  ask,  and  develop  a mindset  that  values  what  we  hear. 
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Knowledge  communities:  a critique  of  current  notions  of  “community” 
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University  of  Montreal 

Ecology  of  meanings:  Communication  and  community. 

The  word  “community”,  as  most  people  know,  comes  from  the  Latin  communitas.  Communitas  means  “being  in 
state  of  communion”.  Etymologically,  the  origin  of  the  word  “community”  carries  the  strong  religious  meaning  of 
“being  connected  with  God”.  It  seems  reasonable  to  state  that  this  word,  in  the  human  and  social  sciences  literature, 
could  be  understood  as  related  to  the  idea  of  “collective”  (Levy,  1974)  or  “connected”  intelligence  (de  Kerckhove, 
1997).  However,  this  “connectedness”  is  difficult  to  define  because  people  can  “connect”  different:  in  a natural  or 
instituted  way.  As  social  beings,  humans  are  dependent  on  each  other  and  nothing  can  be  achieved  in  isolation. 
Following  this  line  of  thinking  I could  state  that  even  when  an  individual  is  eating  his  / her  breakfast  alone,  for 
example,  he  / she  is  connected  to  all  those  who  made  this  meal  possible:  the  farmers  who  planted  the  cereals  and 
those  who  raised  cows  and  chicken,  the  workers  who  did  the  harvest,  milked  the  cows  and  collected  the  eggs,  the 
people  who  transported  those  products  to  the  supermarkets  as  well  as  the  employees  who  put  them  in  the  shelves  and 
sold  them.  This  is,  indeed,  a community  in  the  original  sense:  all  those  people  are  in  a state  of  communion,  are 
connected  (Dalai  Lama,  1999).  I call  such  a community  “natural”  because  the  person  who  eats  the  meal  is  not 
necessarily  communicating  intentionally  but  is  acting  as  a participant  of  a chain  of  causes  and  consequences 
bounding  people  in  the  social  fabric  of  life,  in  which  communication  is  not  being  done  only  through  an  alphabetical 
“language”  but  also  through  media  of  another  nature.  The  most  apparent  media,  in  this  case,  are  money  or  the 
economy  of  food  production  and  consumption,  and  the  power  attached  to  the  public  and  private  bureaucracy  of  the 
organizations  involved  in  the  trade  (Habermas,  1987).  Different  levels  of  actions  are,  thus,  embedded  in  this  process: 
commercial  (the  logic  of  money  exchanges  and  the  power  of  organizations)  as  well  as  symbolic  (the  values  attached 
to  the  food).  Therefore,  this  process  happens  in  an  ecological  context  - material  and  spiritual  - of  the  dialectical 
struggle  for  survival  that  confronts  the  social  groups  that  make,  transport,  sell  and  consume  (or  better  said, 
“mediate”)  eating  objects.  A concrete  example  of  the  above  explained  chain  that  links  communities  is  hurricane 
Katrina  that  wiped  out  the  livelihoods  of  millions  of  people  in  Louisiana,  Mississipi  and  Alabama  (August,  2005). 
The  disruption  of  life  as  it  was  conceived  by  those  people  is  related  to  the  material  lost  of  everything  they  had.  This 
conjuncture,  as  a result,  damaged  communities  not  only  materially:  their  ways  of  communication  became 
increasingly  violent  and  produced  what  the  government  called  “lawlessness”,  when  most  people  living  under  those 
conditions  would  just  call  “survival”.  The  supposedly  lack  of  seriousness  and  responsibility  from  the  part  of  a 
Federal  government  that  seems  to  have  invested  the  material  resources  needed  to  face  domestic  catastrophes  in  Irak, 
lacking  the  will  of  intervening  right  from  the  beginning  in  the  “war”  zone  of  the  Mississipi  delta,  are  some  of  the 
interwoven  material  and  spiritual  aspects  of  a dynamic  ecology  of  meanings. 

In  this  paper,  we  propose  the  notion  of  ecology  of  meanings  as  a theoretical  solution  for  integrating  the  psycho- 
social processes  of  the  human  species  studied  by  the  social  and  human  sciences  into  the  larger  ecology  of  nature  in 
order  to  explain,  in  another  level,  relationships  between  knowledge,  learning,  communication  and  technology.  Every 
ecological  structure  is  composed  of  a number  of  systems  opened  to  the  environment,  functioning  dynamically  and 
permanently  in  the  search  for  equilibrium.  In  the  case  of  food  production  systems  given  above  as  example,  this 
search  for  equilibrium  can  be  found  in  the  provisional  solutions  given  to  problems  by  people,  which  go  on  changing 
along  the  commercial  chain  (which  product  for  which  price  under  which  mark  to  satisfy  what  market,  etc.).  In  the 
case  of  the  survivors  of  hurricane  Katrina  search  for  equilibrium  can  be  found  in  the  numerous  ways  people  tried  to 
survive  (by  evading  the  city  when  possible,  by  “looting”  markets  to  get  food  and  water,  by  crying  their  hopelessness 
to  the  TV  cameras,  etc.),  levels  of  government  tried  to  respond  (the  mayor  swearing  at  the  federal  government  in  a 
late  desperate  cry  aiming  to  get  attention  to  the  poor  people  dying  on  the  streets  of  New  Orleans,  the  president 
asking  for  “patience”  while  recognizing  that  the  help  provided  was  not  “acceptable”  and  defending  his  policies 
telling  the  American  people  that  the  nation  was  sufficiently  wealthy  to  “both”  keep  troops  in  Irak  and  helping  the 
victims  of  hurricane  Katrina),  and  the  resulting  human  and  social  adjustments  being  done  to  deal  with  the 
complexities  of  such  a disaster.  These  human  ecologies  of  problem- solvers  - one  fictitious  and  another  a terrible 
reality  - have  both  (1)  logical  components  (the  logic  of  the  exchanges)  that  structure  how  the  communication 
process  - organic  needs,  money,  business,  food  markets  or  convoys,  power,  etc.  - function  (through  which  medium 
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or  media1),  and  (2)  spiritual  values  that  express  cultural,  religious  and  moral  meanings  concerning  what  each 
context. 

The  “natural”  actions  of  such  communities  (meaning  spontaneous  reactions  to  a chain  of  causes  and 
consequences)  do  not  exclude,  evidently,  the  intentionality  of  the  individual  actors  or  organizations  in  their  problem 
solving  efforts.  A farmer,  for  example,  could  decide  to  lower  the  price  of  a product  which  will  soon  rot  and  need 
immediate  commercialization.  When  the  American  government  decided  that  it  was  reasonable  to  send  help  to  the 
victims  of  the  2005  tsunami  in  Indonesia  in  two  days  while  deciding  to  take  five  days  to  do  the  same  in  Louisiana, 
Mississipi  and  Alabama,  it  took  an  “institutional”  action  that  was  at  the  core  of  how  communities  “naturally”  reacted 
to  the  chain  of  causes  and  consequences  triggered  by  the  landing  of  hurricane  Katrina  in  the  American  South.  These 
actions  do  not  mean,  necessarily,  “awareness”  of  pertaining  to  a community. 

Communities  can  also  be  instituted.  People  can  participate  in  collectives  such  as  churches,  councils,  clubs,  etc, 
“instituted”  by  the  norms  of  organizations.  In  this  regard,  the  advent  of  the  Internet  and  its  “searching”  (intentional) 
possibilities  offers  many  examples  on  how  individuals,  groups  and  organizations  intentionally  “institute” 
communities:  from  educational  institutions  that  impose  online  activities  as  a part  of  the  curriculum  to  individuals 
that  enable  online  groups  through  organizations  (Yahoo,  MSN,  etc.)  such  as  groups  of  adolescents  playing  online 
games  together  from  a distance  or  alcoholic  people  searching  for  the  help  of  others  in  special  websites  offering 
person-to-person  and  / or  collective  support.  All  these  are  indeed  “instituted”  communities,  but  of  another  kind. 
When  people  decide  to  participate  in  those  communities,  the  process  is  not  “natural”  but  “institutionalized”  because 
the  person  who  is  acting  is  necessarily  engaging  in  a previously  given  structure.  Although  one  could  argue  that  the 
same  happens  with  “natural”  communities,  putting  forward  the  argument  that  when  a citizen  is  bound  to  its 
neighborhood  as  a result  of  a choice  of  living  in  it  (and  by  definition  any  choice  would  not  be  “natural”),  the  fact  is 
that  in  any  neighborhood  we  could  find  an  organized  set  of  norms  to  be  necessarily  followed  as  we  found  in  a school 
in  which  students  should  to  this  or  that  (including  using  communication  technologies).  Educational  institutions 
require  through  its  hierarchy  of  power  that  a number  of  activities  be  compulsorily  done  in  order  to  the  school 
authority  have  the  institutionalized  right  to  provide  a school  bulletin  certifying  that  the  student  has  effectively  (for 
example)  communicated  online  and  “learned”. 

Acting  and  communicating  cannot  be  separated  (Piaget,  1977a;  1950;  Habermas,  1987).  Ecological  organic- 
symbolic  systems  such  as  the  human  ones  can  be  more  or  less  natural  or  more  or  less  institutionalized,  provided  that 
we  do  not  find  them  in  pure  forms.  The  notion  of  ecology  that  we  adopt  integrates  logical  and  meaning  systems,  and 
cannot  be  naively  understood  as  strictly  empiricist  or  strictly  informed  by  experience.  Ecologies  of  meanings  are  on- 
going processes  that  have  embedded  empirical  facts  integrated  in  logical  reasons  that  have  possibilities  and  find  its 
actualization  through  communication.  In  the  examples  cited  above,  physical  and  symbolic  action  can  be  considered 
more  “natural”  because,  to  exist,  those  communities,  by  communicating , generate  chains  of  causes  and  consequences 
that  bound  people  together  in  the  social  fabric  more  through  their  subjectivities  (the  phenomenological  living 
experience)  than  through  institutionalized  organizations.  Conversely,  physical  and  symbolic  action  can  be 
considered  less  “natural”  when,  to  exist,  communities,  by  communicating , generate  chains  of  causes  and 
consequences  that  bound  people  together  in  the  social  fabric  under  the  norms  of  institutions  and  the  Law  in  general 
(the  constraints  of  the  social  world).  Such  norms  and  bylaws  (as  imposed  expressions)  are  exercised  (or  not)  through 
personal  “authority”  or  social  codes  that  act  as  instances  of  communicative  media  such  as  “money”  that  activate 
mechanisms  of  “social  control”,  “power  exercise”,  “knowledge  management”,  etc.,  that  subordinate  language 
communication  (Habermas,  1987).  As  in  the  case  of  “institutionalized”  communities,  the  meaning  processes  of 
“natural”  communities  are  triggered  by  dialectical  contexts  of  social  unbalance  which  strike  individuals  to  fight  for 
equilibrium  through  communication.  Ecologies  of  meanings  stand  for  a theoretical  proposal  of  how  communication 
unfolds  in  contextual  unbalanced  systems  that  integrate  the  phenomenological  experience  of  living  with  the 
constraints  of  the  social  world  that  could  be  structurally  represented  as  logic  of  the  exchanges  carrying  spiritual 
values. 


Networked  conviviality:  Participation,  engagement  and  knowledge  building 

Dichotomizing  “logic”  and  “values”  that  evolve  through  communication  medium  or  media  (as  different  as 
money,  power,  speech  and  / or  multimedia  languages)  is  central  for  a ecology  of  meanings  because  it  represents  the 
problem  of  cognitive  structures  vis-a-vis  affective  and  moral  values  attached  to  communicational  exchanges.  Taking 


1 1 use  the  term  “medium”  to  define  one  material  and  / or  symbolic  communication  object  that  carries  operatory  quantities  and 
qualities  of  an  ecological  system  (logical  and  meaning  systems) , and  “media”  to  material  and  / or  symbolic  integrated 
communication  objects. 
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part  of  natural  or  institutionalized  communities  demands  people  to  respond  cognitively  (meaning  “logically”, 
“reasonably”),  affectively  and  morally  to  the  social  demands  of  collective  and  individual  actions.  These  actions  can 
be  ones  of  naturally  being  part  of  a network  of  people,  or  one  of  institutionalized  participation.  Natural  or 
institutionalized  participation  can,  from  the  point  of  view  of  individuals  is  here  called  conviviality.  Conviviality  is 
defined  as  the  process  through  which  communities  intentionally  evolve.  Given  that  natural  and  institutionalized 
participation  cannot  be  found  in  pure  forms  but  are  mostly  one  or  another,  different  levels  of  intentionality  can  be 
found,  leading  to  a grater  or  lesser  social  involvement  by  the  means  of  communication.  Conviviality  involves  (la) 
participation  and  engagement,  and  (2a)  knowledge  building,  which  can  be  either  of  (A)  lower,  (B)  regular  or  (C) 
higher  order  levels. 

The  interrelated  notions  of  (1)  participation  and  engagement  express  the  interests,  needs  and  goals  that  are 
involved  in  the  communication  exchange  within  a community.  Participation  and  engagement  obey  a logic  of 
exchanges  that  dialectically  evolves  upon  time.  This  operatory  logic  of  exchanges  is  the  mechanism  through  which 
interwoven  affective  and  moral  values  that  express  culture  are  communicated.  It  creates  and  / or  re-creates 
knowledge,  which  is  the  content  of  the  logical  operatory  structure.  This  leads  to  the  notion  of  (2)  knowledge 
construction  as  the  co-elaboration  of  thinking  objects  such  as  ideas  and  opinions  that  might  or  might  not  be  carried 
through  arguments.  For  example,  when  a hypothesis  is  formulated  by  someone  (having  or  not  subjacent  the  logical 
operator  “if-then”),  a contestation  by  someone  else  can  result  (based  on  any  other  logical  operator  such  as  “no”, 
“either-or”,  if-then”,  etc.).  Complementarily,  the  content  of  knowledge  construction  is  the  affective  and  moral  values 
of  argumentation  that  takes  place  in  the  communication  process.  When  this  process  takes  place  in  computer- 
mediated  environments,  we  call  it  one  of  “knowledge-building”  (Scardamalia,  2002). 

We  argue  that  networked  conviviality  through  (lnet)  participation  and  engagement  and  (2net)  knowledge 
building  is  a suitable  notion  for  understanding  the  cognitive  and  the  affective  and  moral  values  that  are  in  computer- 
mediated  communication  processes.  These  integrated  notions  allow  the  researcher  to  determine  whether  a 
communication  act  is  mostly  done  in  contexts  like  those  that  Piaget  call  “egocentric”  and  Habermas  “strategic” 
(instrumental  reason,  teleological  acting)  or  mostly  in  “collaborative”  contexts  in  the  sense  given  by  Piagetian 
“cooperation”  (descentration)  and  Habermasian  “inter-understanding”.  This  idea  of  collaboration  as  Piagetian 
cooperation  and  Habermasian  inter-understanding  demands,  necessarily,  a logic  of  exchanges  in  which  consensus  is 
searched  with  a view  of  providing  reasonable  answers  able  to  validate  ideas:  “Interunderstanding  stands  for 
communication  with  a view  of  a valid  agreement .”  (Habermas,  1987,  page  396,  original  italics  preserved,  translated 
by  myself).  (1)  and  (2)  are  applicable  to  any  kind  of  communication,  and  (lnet)  and  (2net)  to  networked 
communication.  The  problem  is  only  one  of  identifying  the  prevalent  level  of  the  exchange  (A,  B or  C). 

In  this  paper,  we  focus  on  the  communication  process  of  networked  communities.  In  the  case  of  communities 
that  are  built  on  the  Internet,  the  distinction  between  teleological  and  communicative  action  is  particularly  useful  to 
understanding  interaction  (through  conferencing  systems,  chats,  etc.)  because  communication,  in  these  cases,  occur 
in  “places”  that  can  be  recorded  in  textual  and  multimedia  databases.  Recording  allows  an  empirical  verification  of 
the  reasons  behind  the  statements  made  by  people  taking  part  of  a communication  exchange  while  the  affective  and 
moral  aspects  can,  at  least,  being  identified2.  These  recordings  are  referred  to  as  objects  of  knowledge  (Popper,  1994; 
Scardamalia,  & Bereiter,  1994).  Such  objects  of  knowledge  cannot  be  taken  as  the  mediation  itself,  be  through  a 
singular  medium  or  plural  media.  The  communicative  actions  through  which  society  and  culture  come  into  existence 
cannot  be  strictly  confined  to  verbal  language , as  Vygotsky  suggested,  or  to  the  idea  of  mediating  object , as 
proposed  by  his  followers  (Luria,  Leontiev,  Cole,  & Engestrom).  There  are  other  circumstances  of  communicative 
action  that  are,  sometimes,  apparently  “languageless”,  such  as  body  language  and  its  regulatory  organic  processes, 
or  the  image  dimension  of  cinema,  television  and  multimedia.  In  addition,  there  are  also  “languages”  related  to 
money  and  power  as  pointed  out  by  Habermas  (1987)  that  can  be  used  as  means  to  achieve  goals.  For  instance, 
money  has  a very  clear  and  concrete  (although  abstract)  language.  How  can  it  be  that  many  people  write  and  speak 
very  little  and  effectively  communicate  power,  control  and  dominance  upon  others?  Communicative  circumstances 
evolve  as  both  organic  and  symbolic  ever-changing  places  of  knowledge  objects  (that  can  be  physical  and  symbolic 
“materials”  such  as  cash  as  well  as  moral,  cultural  and  spiritual  values  that  emerge  as  thoughts)  that  are 
operationalized  along  time  in  social  life  through  communication.  I call  these  physical  and  symbolic  places 


2 Our  research,  done  in  many  different  community  contexts  (educational,  health,  workplace,  cultural,  etc.)  has  shown  that 
affective  and  moral  reasons  for  communicative  behaviour  can  only  be  assessed  through  online  discourse  if  “outspoken”  (i.e. 
clearly  written)  by  the  online  community  participant  (Campos,  2004a;  2004b;  2004c;  2003;  2000;  Campos,  Laferriere,  & 
Harasim,  2001).  Otherwise,  the  researcher  can  only  suppose  what  those  reasons  are,  and  decide  to  use  other  methodological  tools 
to  discover  them  (such  as  interviews). 
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configurations  of  meanings  (Campos,  1998)3.  Configurations  of  meanings  must  be  understood  not  only  as  logical 
systems  (the  cognitive  structures  of  intelligence)  in  which  intelligences  interconnect  through  communication  (Levy, 
1994;  de  Kerckhove,  1997)  but  also  as  the  intertwined  systems  of  meanings  that  define  the  phenomenological 
circumstances  of  people’s  lives,  that  are  attached  to  those  structures,  and  that  evolve  over  time  (idea  of  history)  as  a 
result  of  ecological  interaction.  In  the  human  experience  language  always  implies  many  different  “languages”,  and 
Vygotsky  was  simply  wrong4  in  elevating  verbal  language  as  the  human  mediation  artifact5. 

The  community  mess 

We  tried  to  make  apparent  a number  of  distinctions  in  the  previous  sections  for  a reason.  (1)  First,  following  the 
contribution  of  Habermas,  I propose  understanding  community  participation  as  more  natural  or  more 
institutionalized.  . (2)  Secondly,  concerning  genetic  and  cognitive  psychology  with  regards  to  intentional  processes, 

I distinguish  between  the  unintentional  and  intentional  level  of  mostly  natural  or  institutionalized  networked 
communities  to  propose  a different  approach  for  categorizing  them.  The  risks  of  establishing  such  “levels”  are 
known:  the  discussions  about  consciousness  are  conflicting  because  this  subject  has  not  yet  achieved  any  kind  of 
consensus  among  philosophers  and  cognitive  scientists.  One  of  the  problems  is  the  difficulty  of  integrating  the 
cognitivist  view  with  the  universe  of  values  and  the  role  of  affectivity.  However,  our  proposal  of  “levels”  is 
indicative  and  not  static.  (3)  Thirdly,  following  genetic  epistemology,  we  distinguish  between  the  logical  mechanism 
of  exchanges  that  operates  moral  (normative  knowledge)  and  / or  cultural  (psycho-social  historical  knowledge) 
values  for  didactic  reasons  though  we  know  that  they  are  complementary  in  the  living  process  of  human  beings. 
Genetic  epistemology  might  not  be  “fashionable”  nowadays  but  it  is  coherent,  something  that  most  contemporary 
empiricist  cognitive  theories,  applied  to  networked  mediated  communication,  are  not.  (4)  Finally,  I distinguish 
participation  / engagement  and  knowledge  building  as  complementary  and  co-implicative  notions  that  allow  us  to 
understand  networked  community  processes  epistemologically  and  pragmatically:  their  rationales  should  be 
understood  as  evolutionary  theoretical  praxis6. 

These  four  distinctions  allows  to  unveil  the  precarious  state  of  reflection  about  community  processes  that  led  to 
the  mainstream  use  of  a sort  of  unofficial  classification  (that  sounds  as  a professional  categorization)  to  define 
communities  of  “interest”,  “learning”  and  of  “practice”.  These  notions  have  been  used  in  the  context  of  the 
increasing  application  of  technology  in  human  communication.  The  idea  of  community  of  interest  has  been  around 
for  some  time  (Clodius,  1996;  1997)  and  has  its  origins  in  the  early  works  of  Rheingold  about  virtual  communities. 
Rheingold  defines  communities  as  “social  aggregations  that  emerge  from  the  Net  when  enough  people  carry  on 
those  public  discussions  long  enough,  with  sufficient  human  feeling,  to  form  webs  of  personal  relationships  in 
cyberspace”  (2000,  second  edition,  page  XX).  Clodius,  referring  to  participants,  adds  “shared  interests  and  self- 
identification  of  belonging  to  a group”  (1997).  This  sense  of  purpose  - the  “interest”  - is  at  the  core  of  the  idea  of 
any  state  of  communion.  People  gather  together  in  communities  on  the  base  of  common  interests,  developed 
progressively  in  the  communication  process.  The  reader  will  soon  realize  that  the  other  “definitions”  of  community 
are  just  this. 

The  notion  of  learning  community  was  introduced  in  education  (and  then  spread  out  to  other  fields)  and  its 
genesis  can  be  hardly  traced.  It  is  certain,  though,  that  its  adoption  by  so-called  “socio-constructivists”  was  a 
reaction  to  a renewed  effort  to  respond  to  traditional  teaching  processes  based  on  the  functionalist  paradigm  of 
information  transmission.  Researchers  and  educators  using  this  term  have  usually  in  sight  the  process  of  co- 
development happening  in  the  ZPD  - zone  of  proximal  development  described  by  Vygostky  (1979)7.  The  use  of  this 


3 In  my  doctoral  thesis  I used  the  term  symbolic  configuration.  Configuration  is  a term  than  implies  itself  the  idea  of  logical 
structure.  I am  using  now  the  predicate  of  meanings  instead  of  symbolic  because  the  first  appears  to  be  more  in  line  with  the 
human  phenomenological  experience  while  the  latter  is,  most  of  the  times,  arbitrary  (ex:  the  American  flag  as  a symbol). 

4 Or,  to  be  fair  and  grateful  for  his  contribution  to  knowledge:  “partially  right”. 

5 1 understand  this  idea  as  being  in  line  with  that  of  a new  instance  of  communication  of  the  collective  intelligence  in  the 
knowledge  society  that  is  being  presently  proposed  by  Pierre  Levy,  and  that  advances  significantly  the  way  people  have  been 
thinking  about  communication.  (Levy,  P.  Personal  communication,  Laboratory  on  Collective  Intelligence,  University  of  Ottawa, 
27  October  2004). 

6 I would  like  to  remind  the  reader  that  praxis  in  Plato  was  bios  theoreticos,  or  the  praxis  of  the  philosopher.  When  community 
discussions  are  at  stake,  it  is  rather  paradoxical  to  formulate  that  the  core  of  their  activity  is  practice  given  that  the  referents  of 
discourse  (what  is  said,  discussed,  debated)  are  reasons,  i.e.  abstractions  and  not  the  objects  themselves  or  their  relationships. 

7 The  notion  of  ZPD  was  conceived  to  explain  the  leg  between  the  child’s  actual  developmental  level  and  the  potential 
developmental  level  that  the  child  could  achieve  with  the  help  of  older  children.  It  is  fundamental  to  understand  that  Vygotsky 
never  used  the  idea  of  ZDP  to  refer  to  adulthood. 
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notion  led  educators  and  researchers  in  the  search  for  renewed  pedagogies  to  focus  on  social  rather  than  on 
individual  processes,  to  defend  context-based  approaches  to  education  (situated  learning)  and  reject  universals.  The 
multiple  “socio-constructivist”  approaches  developed  after  Vygotsky’s  death  add  to  the  already  vague  model 
proposed  by  him  (and  that  became  vaguer  with  the  interference  of  other  group  members  of  the  Moscow  School  in 
his  writings  and  the  difficulty  of  validating  the  meaning  of  a number  of  notions  presented  in  the  translations  done 
thereafter).  Although  Vygotsky  could  not  be  described  as  a radical  revolutionary,  his  interpretation  of  dialectical 
materialism  was  not  in  opposition  with  main  communist  ideas  of  his  time.  However,  it  seems  that  his  view  has  been 
washed  out  from  its  subjacent  revolutionary  praxis , becoming  just  “situated”,  reinforcing  the  typical  a-criticism  of 
neo-behaviorist  cognitivism8. 

The  notion  of  communities  of  practice  resulted  from  one  of  the  socio-constructivistic  re-interpretations  of 
learning  according  to  which  it  “is  a process  of  participation  in  communities  of  practice,  participation  that  is  at  first 
legitimately  peripheral  but  that  increases  gradually  in  engagement  and  complexity”  (Lave,  & Wenger,  1991).  This 
re-interpretation  was  further  developed  by  Wenger  to  explain  organizational  processes  and  developed  as  a theory  of 
social  learning.  According  to  Wenger,  communities  of  practice  are  “a  locus  of  engagement  in  action,  interpersonal 
relations,  shared  knowledge,  and  negotiation  of  enterprises. . . (that  are). . .the  key  for  real  transformation  - the  kind 
that  has  real  effects  on  people’s  lives”  (1998,  page  85).  Such  a generic  definition  is  not  at  odds  with  that  of 
communities  of  interest  and  that  of  learning  communities,  which  makes  the  distinction  entirely  meaningless. 

It  is  important  to  discuss  the  notion  of  praxis  in  Wenger  (whatever  it  is)  because  his  theory  became  very  well- 
known  and  widespread  used  on  the  basis  of  an  epistemological  mess.  The  interest  on  this  notion,  according  to  him, 
resides  on  the  wish  to  present  counterarguments  to  cognitive  scientists  that  do  not  take  into  account  as  fully  as  he 
would  like,  social  practice  as  “the  key  to  grasping  the  actual  complexity  of  human  thought  as  it  takes  place  in  real 
life”  (Wenger,  1998,  footnote  6,  page  281).  Wenger  discusses  praxis  very  briefly  by  drawing  from  Marx  to 
Wittgenstein.  He  easily  jumps  from  author  to  author  without  ever  solving  critically  some  serious  epistemological 
conflicts.  He  lists,  among  several  others,  Marx,  Bourdieu,  Engestrom,  Habermas,  Vygostky  and  Wittgestein,  as 
having  influenced  his  theory.  He  also  lists,  concerning  the  “situatedness”  of  learning,  phenomenologist  and 
empiricist  authors  alike  without  providing  a serious  critical  theoretical  integrative  explanation.  One  example  of  this 
mess  comes  from  authors  cited  by  Wenger:  Cole  and  Engestrom.  Although  the  subjacent  notion  of  praxis  used  by 
these  so-called  vygotskian  socio-constructivists  (Salomon,  1993)  seems  to  be  in  line  with  the  critique  presented  in 
the  first  thesis  about  Feuerbach  (text  in  which  Marx  states  that  the  main  default  of  materialist  approaches  is  that  they 
do  not  take  into  account  human  activity  as  praxis ; Marx,  & Engels,  1982),  the  activity  system  theory  proposed  by 
Cole  and  Engestrom,  based  on  reinterpretations  by  followers  of  Vygotsky,  has  no  revolutionary  purpose  whatsoever. 

In  his  proposal  of  communities  of  practice,  Wenger  writes  that  they  have  “real  effects  on  people’s  lives”  (1988, 
page  85).  It  relies,  surprisingly,  on  psychological  effects  and  not  on  social  effects,  which  seems  not  to  be  consistent 
with  the  premise  of  his  theory,  which  does  not  consider  - at  least  implicitly  - the  possibility  of  any  kind  of  strictly 
individual  learning.  Although  social  contexts  are  always  present,  some  kinds  of  learning  are  indeed  totally 
individual.  For  example:  if  a person  learns  how  to  ride  a bike  alone,  although  we  could  call  the  bicycle  the  “social”, 
this  learning  is  not  social  at  all.  Wenger  also  writes  that  communities  of  practice  “hold  the  key  to  real 
transformation”  (1988,  page  85).  Given  that  “communities  of  practice”  have  been  mostly  implemented  in 
companies,  this  statement  sounds  not  only  pretentious  but  a lie.  Any  transformation  washed  out  of  practices  able  to 
change  structures  of  authority  (power)  within  the  capitalistic  enterprise  and  the  system  that  produces  inequalities 
among  employees  cannot  carry  such  a name.  Such  a “transformation”  does  not  promote  real  but  cosmetic  change,  or 
strategic  change9.  This  theory  is  now  being  used  by  companies  to  keep  themselves  up  and  running  while  employees 
build  the  illusion  that  they  could  matter  as  much  as  the  CEO  does,  which  is  a practice  of  deceit.  It  would  be  naif  to 
believe  in  transformations  carried  out  by  “communities  of  practice”,  which  are  just  institutionalized  endeavours  in 
which  subjectivity  is  caged  and  inter-understanding  is  subordinated  to  instrumental  goals.  Admitedly,  at  a micro 
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I must  clearly  state  here  that  in  spite  of  the  general  understanding  that  the  Cognitive  Sciences  are  in  total  opposition  with 
behaviourism  due  to  the  belief  that  mental  processes  can  be  scientifically  studied,  they  adopt  the  experimental  paradigm  as  their 
main  “objective”  method  of  assessing  thinking  processes.  I believe  that  a theory  shows  its  real  epistemology  through  its  method 
and  not  through  its  declaration  of  principle  (which,  in  the  case  of  the  Cognitive  Sciences,  was  one  of  proclaiming  themselves  as  a 
movement  against  behaviourism).  Most  socio-constructivists  never  rejected  entirely  Cognitive  Science  research  methods , 
integrating  them  with  qualitative  research,  arriving  to  an  unsustainable  empiricist  epistemological  compromise  between 
“universal”  and  “situated”  learning  (Anderson,  J.  R.,  Reder,  L.  M.  et  Simon,  H.  A.,  1996;  Greeno,  J.  G.,  1997;  Anderson,  J.  R., 
Reder  L.  M.  et  Simon,  H.  A.,  1997).  Such  a compromise  is  at  odds  with  the  Kantian-Hegelian  philosophical  tradition  of  dialectic 
materialism,  and  therefore,  most  self-announced“socio-constructivists”  are  just  incoherent  empiricists. 

9 Wenger,  McDermott  and  Snyder  (2002)  have  recently  assumed  that  communities  of  practice  are  to  be  used  strategically  by  the 
organisation  for  their  benefit  but  considering  the  “learning  benefit”  for  the  participants  as  essential  for  their  existence. 
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level  small  enterprises  collectively  ran  and  collectively  owned  could  be  less  cynical  in  the  integration  of  such  tool. 
Wenger’s  notion  of  praxis  is  certainly  not  the  same  that  of  Cole  and  Engestrom  and  that  of  Vygotsky10.  It  is  not, 
either,  in  line  with  the  epistemological  standings  of  authors  as  distant  one  from  another  as  Wittgenstein  and 
Habermas  (although  the  latter  takes  into  account  some  considerations  of  the  Philosophical  Investigations  in  his 
critique  of  functionalist  reason)11.  Wenger’s  epistemological  status  is,  in  fact,  unclear.  Result:  the  notion  of 
community  of  practice  as  a social  learning  theory  is  untenable.  It  is  not  a theory  because  it  has  no  method  and  a 
flawed  epistemological  foundation. 

Social  scientists,  differently  from  pedagogues,  provided  more  consistent  theories.  One  example  is  Habermas 
theory  of  communicative  action  because  it  merges  the  phenomenological  dimension  of  existence  to  the  workings  of 
the  social  system  through  communication,  evolving  due  to  learning  as  understood  by  Piaget,  taking  critically  into 
consideration  both  achievements  of  the  natural  and  the  social  sciences.  The  introductory  discussion  on  communities 
presented  above  reveals  the  difficulty  to  integrate  such  processes  from  a psychological  and  sociological  perspective, 
relating  them  to  ontogeny  and  phylogeny.  Summarizing:  If  people  “learn”  in  communities  of  “interest”  then  this 
community  is  also  a learning  one.  If  they  establish  a “practice”  of  negotiation  of  meanings  then  they  are  also  a 
community  of  practice  (understood  as  bios  theoreticos).  If  people  involved  in  a learning  community  have 
“interests”,  then  this  should  be  considered  a community  of  interest.  If  this  “learning”  community  has  “practices”  of 
engaging  in  common  activities,  then  it  is  a community  of  practice  that  “learns”  or  a learning  community  that  has  a 
“practice”.  To  be  engaged  in  a common  “practice”,  people  must  have  an  “interest”  as  well,  and  then  a community  of 
practice  should  also  be  named  a community  of  interest.  This  mess  leads  anywhere.  We  want  to  provide  here  a 
glimpse  of  a tentative  integrative  solution  for  this  problem  applicable  to  the  traditions  of  both  the  social  (sociology, 
anthropology,  political  science  and  communication)  and  the  human  sciences  (psychology  and  education)  by  trying  to 
build  a communication  theory  based  on  Habermas  and  Piaget. 

Knowledge  as  the  distinctive  character  of  communities 

We  propose  categorizing  communities  from  the  point  of  view  of  knowledge,  relying  on  the  evolution  of  the 
Kantian  and  Hegelian  tradition  that,  in  the  social  sciences  led  to  the  theory  of  communicative  action  (Habermas), 
and  in  the  human  sciences  led  to  epistemological  constructivism  (Piaget).  I argue  here  that  these  traditions  are  the 
result  of  specific  scientific  trajectories  that  have  more  in  common  than  apparently  one  could  suppose.  By  pointing  to 
this  theoretical  parallelism  (already  highlighted  by  Habermas  concerning  Piaget’s  epistemology),  we  would  like  to 
defend  the  thesis  that  it  is  through  configurations  of  meanings  that  the  mind,  through  its  (multi-language) 
communicative  action,  deals  with  its  body,  its  phenomenological  subjectivity,  and  the  culturally  anchored  norms  of 
the  social  world.  Human  ecological  systems  are  organic  and  mental,  and  fundamentally  meaningful.  On  another 
dimension,  the  symbolic  processes  of  interaction  among  configurations  of  meanings  communicated  by  different 
minds  (rational  assimilation  and  accommodation  of  meanings  leading  to  adaptation,  i.e.  conceptual  change)  are  an 
integrative  but  logically  anterior  part  (or  a moment)  of  the  praxis  of  the  human  species.  Reason  structures  passions 
and  reflections  about  moral  behavior,  and  not  the  contrary12.  We  argue  that  this  understanding,  foundation  of 
ecology  of  meanings  as  a theoretical  proposition,  can  lead  to  acceptable  arguments  to  treat  culture  and  nature 
altogether  by  the  means  of  deepening  our  understanding  on  the  unique  communication  found  in  the  human 
experience  and  the  resulting  forms  of  communities. 

We  proposed  elsewhere  (Campos,  2003)  one  taxonomy  of  knowledge  networked  communities  that  would 
overwrite  the  notions  of  communities  of  interest,  learning  and  of  practice,  in  the  context  of  an  evolutionary  ecology 
of  meanings.  This  taxonomy  results  from  research  and  reflections  about  community  experiences  using  methods  such 
as  discourse  analysis,  ethnographic  intervention,  and  clinical  observation.  Although  this  classification  could  be 
applied  to  any  kind  of  communication  context,  our  focus  here  is  on  communication  mediated  by  computer 
technologies.  We  believe  that  this  classification  is  a first  development  towards  a theory  in  line  with  the  integrative 
revision  of  critical  theory  proposed  by  the  Habermas  of  the  theory  of  communicative  action  in  which  rationality  is 


10  The  model  of  behaviour,  according  to  Vygostky  is  from  the  Stimulus  to  the  Response  (S— >R),  mediated  by  language  (L).  The 
activity  theory  proposed  by  Leontiev  replaced  S by  Subject  and  R by  Object,  linking  both  by  a bidirectional  arrow  (S<->0) 
mediated  by  MA:  mediating  artefact.  This  was  the  basis  for  Cole’s  and  Engestrom’ s work.  The  serious  epistemological  (and 
political)  results  of  this  change  are  obvious  because  the  model  proposed  by  Vygotsky  has  an  implicative  cause-consequence 
leitmotiv  (— >)  that  could,  according  to  Marxism,  be  interpreted  differently. 

11  Although  the  contributions  of  Wittgenstein,  a logical  positivist,  to  the  study  of  language  are  extraordinary,  one  should  ask  how 
his  philosophical  reflections  on  the  empiricist  basis  of  meanings  could  be  in  line  with  the  critical  view  of  the  rationality  of  action 

developed  by  Habermas. 

12 

In  fact,  the  inversion  is  seen  as  pathologies. 
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studied  through  a renewed  discourse  analysis  that  goes  beyond  Austin’s  and  Searle’s  empiricism,  and  that  it  is  more 
adequate  to  the  study  of  communities  than  the  notions  of  interest,  learning  and  practice.  We  have  been  analyzing 
discourse  based  on  the  Piagetian  sociological  proposal  of  the  logic  of  exchanges  that  partially  inspired  Habermas  in 
his  conceptualization  of  communication  as  a core  instance  of  rationality13,  relating  levels  of  participation  and 
engagement,  and  knowledge  building.  We  identified  three  main  epistemological-communicative  levels  of 
progressive  relationships  that  can  be  found  in  communities  that  we  call  “of  knowledge”14: 

Co-presence 

Co-presence  is  when  people  engage  in  contexts  in  which  the  actors’  intentionality  is  mainly  egocentric  or  a 
result  of  imposition  or  trade.  The  result  is  representations  that  are  mainly  individual,  inter-personal  relationships  that 
are  established  through  interactions  in  the  community  that  do  not  go  beyond  contact  making  (“hi  there”),  and 
learning  that  does  not  go  beyond  assimilation.  Learning  is  a complex  process  of  adaptation  that  implies  assimilation 
and  accommodation  of  concepts  (Piaget,  1977b).  In  co-presence  there  are  weak  communicative  meaning 
configurations  (because  there  are  no  or  meaningless  responses)  able  to  produce  the  necessary  friction  to  move 
participants  towards  debate  (argumentation),  thought  evolution,  and  search  for  inter-understanding.  In  such  a 
context,  the  level  of  communication  characteristic  of  a community  could  be  classified  as  low,  given  that  the 
exchanges  do  not  trigger  further  reflection,  leading  to  a valid  accord  among  its  members.  The  level  of  co-presence 
characterizes  what  we  could  call  the  publishing  mode.  Communities  of  “interest”  such  as  those  built  around  most 
discussion  lists  in  which  people  just  publish  information  and  hyperlinks  and  “talk  to  themselves”,  “learning” 
communities  such  as  those  built  around  discussion  forums  used  by  professors  to  inform  students  about  course 
content  in  which  no  discussions  are  made  and  few  comments  are  published  without  response,  and  communities  of 
“practice”  in  companies  in  which  most  messages  are  used  just  to  draw  the  attention  of  others  on  a given  subject, 
could  all  be  classified  as  having  a lower  level  of  knowledge  building.  Relationships  in  these  publishing  or 
broadcasting  communities  are  ones  of  co-presence  and  nothing  else.  They  could,  at  the  most,  also  be  classified  as  a 
teleological  because  instrumental  reason  (egocentric,  strategic)  is  what  is  mostly  at  stake. 

Cooperation 

In  a networked  context  of  cooperationl5,  intentionality  is  still  mainly  egocentric  or  a result  of  a context  of 
imposition  or  trade.  However,  the  self-centered  goals  of  the  community  actors  are  reached  through  activities  that 
engage  more  clearly  the  other  participants.  As  a consequence  of  this  interaction,  the  configurations  of  meanings  that 
impulse  the  community  into  the  practice  of  discussing  - as  always  is  the  case  with  our  minds  - have  at  times  more 
“individual”  goals  and  at  other  times  more  “social”  goals.  However,  intentions  remain  mainly  personal  and 
egocentric,  or  instrumentalized  to  achieve  the  goal  defined  by  the  imposing  authority  or  norms  of  the  trade.  In  this 
case,  participants  are  not  really  interested  in  helping  one  another  but  to  achieve  set  goals.  Interpersonal  relationships 
are,  thus,  “cordial”  but  most  of  the  times  competitiveness  is  what  is  really  at  stake.  Such  kind  of  “social”  learning, 
built  in  cooperative  situations  is  one  in  which  the  individual  is  still  bound  to  more  assimilatory  than  accommodative 
processes,  and  if  - and  when  - accommodation  happens,  the  resulting  conceptual  change  is  not  shared  and  cannot  be 
found  in  the  discourse  (it  rests  in  the  mind  of  the  participating  individual  and  hence  can  not  be  traced  through 
discourse  analysis  but  only  through  other  methods).  Cooperation  characterizes  the  collegial  mode.  Communities  of 
“interest”  such  as,  for  example,  those  in  which  participants  search  for  information  about  an  illness  and  get  it  from 
other  participants,  “learning”  communities  such  as  those  built  around  discussion  lists  used  by  professors  in  which 
teams  of  students  have  to  answer  appropriately  some  questions  about  the  course  content  to  get  points,  and 
communities  of  “practice”  in  organizations  in  which  employees  discuss  the  use  of  the  most  appropriate  materials  to 
be  used  in  a road,  for  example,  could  all  be  classified  as  having  a medium  level  of  knowledge  building. 

Relationships  in  such  cooperative  communities  are  collegial.  They  could  also  be  classified  as  a teleological  as  well 
because  instrumental  reason  (egocentric,  strategic)  is  what  is  mostly  at  stake. 
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I would  like  to  thank  my  graduate  students  Jonathan  Petit  and  Mathieu  Chaput  who  drew  my  attention  to  Habermas. 

14  The  method  for  identifying  the  levels  is  based  on  analyses  of  progressive  discourse  built  on  the  Piagetian  logic  of  exchanges  in 
which  logical  forms  subjacent  to  human  discourse  are  considered  together  with  affective  and  moral  values  (Campos  and  Tomi, 
2004;  Campos,  2004). 

15  Although  the  choice  of  using  the  word  cooperation  in  this  context  could  be  confusing  for  Piagetians,  I do  not  adopt  it  in  the 
sense  of  Piaget.  Piagetian  cooperation  is  collaboration  in  this  proposal,  thus  the  next  level  as  the  reader  will  soon  realize. 
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Collaboration 

Collaboration  is  achieved  when  community  participants  engage  intentionally  in  joint  efforts  to  achieve 
common  understanding  of  a given  subject  matter.  Participants  solve  problems  together  to  advance  the  present  state 
of  knowledge  not  for  themselves.  Intentionality,  in  this  case,  is  mostly  centered  on  the  others  and  not  on  the  self  or 
on  a strategic  goal.  The  solution  is  to  be  discovered  somewhere  in  between  the  participants’  configurations  of 
meanings.  As  a result,  even  if  individual  representations  are  a material  for  building  configurations  of  meanings  - as 
they  will  always  be  - the  object  of  discussion  is  essentially  built  around  what  representations  could  be  built  together 
(social).  In  such  a situation,  interpersonal  relationships  suppose  honest  and  authentic  mutual  implication.  In  this 
case,  participants  go  beyond  heteronomy16  and  authority,  if  exercised  (be  that  of  money,  position,  status,  etc.),  is 
dissolved  or  diminished  in  an  inter-understanding  process  in  search  of  a valid  rational  agreement  among 
participants.  In  autonomous  processes  like  these,  people  accept  the  possible  inequality  as  not  relevant  to  the  purpose 
(Piaget,  1977).  Community  processes  in  situations  of  autonomy  (democracy)  are  necessarily  shared,  the  resulting 
mainly  accommodatory  learning  is  collectively  built,  and  authorship  of  ideas  impossible  to  be  claimed  by 
individuals  (Scardamalia,  & Bereiter,  1994;  Scardamalia,  2002).  A learning  process  leading  to  conceptual  change  is 
necessarily  accommodatory:  participants  engage  in  in-depth  inter-understanding,  co-interpretation  and  co- 
construction of  explanations  concerning  ideas  that  emerged  in  the  communication  process.  At  this  level, 
communities  achieve  higher  levels  of  communication  through  collaboration,  characterizing  the  knowledge  building 
mode  of  interpretive  or  knowledge  building  communities  or.  Communities  of  “interest”  such  as  the  WELL  referred 
by  Rheingold  (2000)  had  moments  of  true  collaboration.  “Learning”  communities  and  communities  of  “practice” 
that  are  institution  independent  (or  that  are  able  to  go  beyond  the  constraints  of  authority  within  institutions,  ruling 
their  processes  independently  of  external  constraints)  can  also  achieve  a state  of  authentic  knowledge  building  if 
they  go  beyond  authority  as  a means  to  transform  it  (“revolution”).  Only  at  this  level,  in  which  partners  achieve  a 
process  of  what  Habermas  (1987)  calls  inter-understanding  (or  Piagetian  “cooperation”17)  communities  would  be 
exercising  communicative  reason  in  order  to  achieve  valid  agreements  based  on  consensus  or  accepted  majority,  and 
able  to  critically  transform  the  world  through  democratic  means.  They  could  be  classified  as  democratic 
communities. 


Conclusion 

We  presented  a proposal  based  on  an  integrative  view  of  cognitive  and  social  studies  aiming  to  explain 
communication  processes  as  related  to  communities.  The  theoretical  proposal  of  ecology  of  meanings  defends  that 
both  communication  and  resulting  communities,  as  the  core  of  the  meaning  human  experience,  can  be  better 
understood  as  ecologies  that  evolve  historically  solving  perpetually  the  problems  posed  by  the  existing  apparent 
contradictions  between  the  phenomenological  experience  of  living  and  the  demands  of  locating  in  and  adjusting 
bodies  and  minds  to  the  rules  and  regulations  of  society.  To  make  our  point,  we  introduced  a draft  of  a critique  (that 
we  intend  to  develop  more  extensively  in  future  writings)  about  notions  of  community  that  are  being  used  nowadays 
that  we  consider  insufficient  and  inadequate  as  classificatory  categories.  In  addition,  we  discussed  epistemological 
conflicts  subjacent  to  those  categories,  and  proposed  that  communities  be  classified  according  to  the  level  of 
knowledge  building  and  communication:  publishing  or  broadcasting,  and  collegial  communities  that  mostly  share 
egocentric  and  teleological  functionalist  reasons,  and  interpretive  or  knowledge  building  communities  that  mostly 
share  communicative  reason. 


16  Heteronomy  stands  for  situations  in  which  there  is  a clear  ruling  authority  while  autonomy  stands  for  situations  in  which  power 
is  democratically  shared. 

17  See  note  15. 
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Cueing  Strategies  and  Instructional  Message  Design 

Yu  Cao 

Virginia  Polytechnic  Institute  & State  University 

In  computer-based  instruction,  the  way  the  computer  screen  is  perceived  and  the  way  the  information  on  it 
is  presented  can  impact  its  instructional  effectiveness  (Card,  Moran,  & Newell,  1983;  Grabowski,  1995;  Winn, 

1993).  There  is  a trend  that  computer  will  be  involved  more  in  the  teaching  and  learning  process  since  the 
“demographic  changes  among  students,  rising  education  costs,  and  new  technology”  (Frey,  Yankelov,  & Faul,  2003, 
p.  443).With  the  advanced  integration  of  computer  in  the  teaching  and  learning  process,  an  issue  about  how  to 
design  and  deliver  instructional  message  on  the  computer  screen  starts  to  cause  professionals’  attention.  Instructional 
designers  must  consider  how  message  form  and  structure  influence  information  process  (Winn,  1993)  and  make  the 
instructional  materials  minimize  cognitive  processing  burdens  so  that  mental  efforts  spent  on  processing  task 
relevant  information  so  as  to  enhance  learning  performance.  Researchers  (Grabowski,  1995;  Winn,  1993)  emphasize 
that  the  form  of  the  message  affects  what  information  is  attended  to  and  perceived  by  learners,  especially  in  the  early 
stage  of  information  processing.  Gagne’s  (1985)  theory  regarding  the  condition  of  learning  suggests  that  the  internal 
processes  of  learning  can  be  influenced  by  external  events.  Dick,  Carey,  and  Carey  (2001)  also  address  that  in 
instructional  design  model,  using  cues  to  search  the  information  which  was  stored  in  memory  is  one  of  the  essential 
functions  for  the  condition  component  of  behavior  objectives.  The  provision  of  cueing  strategies,  as  external  events, 
may  direct  the  attention  of  the  learner  by  providing  clues  as  to  what  aspects  of  the  materials  to  attend  to  in  a 
succeeding  learning. 

Introduction  of  Cueing  Strategies 

Researchers  have  begun  to  identify  the  elements  of  the  message  which  are  important  in  facilitating  learning 
achievement  during  instructional  message  design  process  (Lamberski  & Mayers,  1980).  Studies  of  message  design 
have  suggested  that  the  message  should  be  clearly  defined  and  set  forth  to  provide  optimum  instructional 
effectiveness  and  efficiency  in  the  learning  process  (Lamberski  & Mayers,  1980).  In  order  to  guarantee  the  efficient 
learning  to  occur,  “. . . the  stimulus  material  for  learning  must  be  available  and  clearly  perceivable  by  the  learner” 
(Dwyer,  1978,  p.  158).  Instructional  design  must  find  a way  to  magnify  or  emphasize  the  critical  features  in  the 
instructional  message  (M.  Fleming  & Levie,  1978).  Previous  researchers  (Ausubel,  1968;  Rothkopf,  1966)  found 
that  positive  learning  was  affected  by  using  strategies  to  direct  learners’  attention  to  relevant  information.  Since 
learning  from  computer  screen  delivered  message  is  basically  an  individual  or  independent  type  of  learning 
experience,  attention  directing  learning  strategies  should  help  learners  to  identify  relevant  information  which  should 
be  learned. 

Inserting  cues  in  the  instructional  text  provides  a mechanism  for  helping  learners  to  organize  complex 
information,  since  the  use  of  cues  can  direct  or  focus  students’  attention  on  areas  that  are  important  to  the 
instructional  goals  and  objectives  (Dwyer,  1978).  This  type  of  instructional  strategies  has  been  shown  to  compensate 
for  students  who  have  perceptual  deficiencies  in  comprehend  visual  information  (Dwyer,  1978).  Flemming  and 
Levie  (1978)  stated,  “one  of  the  most  obvious  manipulations  of  the  learning  environment  is  that  of  modifying  the 
instructional  stimuli  to  be  presented”  (p.  112).  One  of  the  principles  they  proposed  of  controlling  attention  in 
instructional  situations  is  that  “instructional  messages  can  control  the  effective  stimulus  either  by  adding  or 
subtracting  cues  or  by  making  cues  more  or  less  salient  (prominent,  noticeable)”  (p.  112).  They  believe  that 
“learning  is  facilitated  where  critical  cues  are  salient  (dominant,  apparent,  conspicuous)  (M.  Fleming  & Levie,  1978, 
p.l  15).  They  also  points  out  that  if  cues  which  were  added  in  the  instructional  message  were  familiar  to  the  learners 
and  they  functioned  to  direct  attention  to  the  relationship  between  the  new  knowledge  and  the  previous  knowledge, 
they  can  facilitate  learning. 

Researchers  have  defined  cueing  from  different  perspectives.  Rieber  (1994)  views  cueing  as  any  action  that 
directs  a learner’s  attention  to  the  important/relevant  stimuli.  Goldstein  (1981)  describes  cueing  as  a type  of 
rehearsal  strategy  which  involves  the  presentation  of  a word  associate  to  be  utilized  for  the  retrieval  of  items  in 
recall.  Turcott  (1986)  defined  cueing  as  a technique  whereby  individual  stimuli  or  elements  within  a visual  array  are 
made  distinct  from  each  other.  He  stated  that  a cueing  device  refers  to  the  specific  attribute  such  as  color,  arrows,  or 
shading  which  is  used  to  make  specific  element  distinct  to  the  viewer.  Partee  (1984)  defines  a cue  as  “a  verbal  or 
visual  strategy  by  which  the  perceiver’s  attention  is  focused  upon  perceptual  information  (p.  12).  Dwyer  (1978)  has 
defined  cueing  as  “the  process  of  focusing  learner  attention  on  individual  stimuli  within  the  illustration  to  make  the 
essential  learning  characteristics  distinct  from  other  stimuli”  (p.  158).  Researchers  (Brown,  1978;  Brown,  Lewis,  & 
Harcleroad,  1973)  in  instructional  message  design  have  named  cues  in  instruction  as  indicating  cues  (e.g.  arrows, 
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directions  to  pay  attention  to,  to  notice  certain  parts  of  an  instructional  sequence,  and  the  use  of  color  codes,  etc.). 
Brown  (1978)  mentioned  that  the  indicating  cues  have  commonly  referred  to  in  the  literature  as  ‘attention-directing 
techniques’  or  merely  cues.  Cues  also  have  been  named  as  attention-getting  devices  (Norman,  1969)  or  attention 
gaining  technique  (Dwyer,  1978).  F.  M.  Dwyer’s  definition  of  cueing  emphasizes  more  on  the  effects  of  cues 
utilized  in  visualized  instruction,  while  this  article  intends  to  discuss  cues  in  computer  screen  delivered  text 
message.  Therefore,  the  definition  that  employed  in  this  study  will  be:  cueing  strategy  is  the  attention-directing 
strategy  to  focus  learners’  attention  on  specific  information  within  text  message  on  computer  screen  so  as  to  improve 
students’  learning  performance  in  computer-based  instruction. 

Cues  have  been  categorized  as  verbal  cues  and  visual  cues  (T.  H.  Anderson  & Armbruster,  1985).  Verbal 
cues  include  instructional  objectives,  advance  organizer,  adjunct  questions,  and  signal  phrases.  Visual  cues  include 
typographic  cuing  achieved  by  manipulating  the  type  face,  type  styles,  underlining,  capitals,  color  of  text,  etc.  Allen 
(1975)  summarized  the  literature  on  visual  cues  and  reported  that  “relevant  cues  which  emphasize  material  to  be 
learned  within  an  instructional  communication  may  increase  the  learning  of  that  material  for  all  [intellectual]  ability 
group.”  (p.  151).  Dwyer  (1972;  1978)  states  that  visual  cues  are  attention-directing  techniques  used  to  focus  the 
learners’  attention  on  certain  stimuli  or  the  critical  aspects  of  instructional  material. 

Bloom  (1976)  provides  several  important  views  about  cues  for  instructional  designers.  First,  cues  have  to 
do  with  their  meaningfulness.  The  cues  must  be  understood  and  comprehended  by  the  learners.  If  cues  are  familiar 
and  have  been  used  by  the  learners  before,  they  are  more  likely  to  be  easily  learned  in  a new  context.  Secondly, 
learners  may  different  in  learning  from  particular  cues,  such  as  some  students  are  easily  to  learn  using  verbal  cues  or 
others  may  prefer  to  visual  cues.  Researchers  and  instructional  designers  have  been  spent  much  attention  on  finding 
the  best  “one”  form  of  presenting  cues  to  facilitate  learning.  Finally,  Bloom  shows  that  the  trend  of  the  development 
of  new  technologies  might  provide  variety  of  instructional  methods  to  help  learners  to  secure  the  cues  they  need  for 
their  learning. 


Functions  of  Cueing  Strategies 

Dwyer  (1978)  reported  that  the  ability  to  focus  attention  on  relevant  cue  is  fundamental  to  learning  that 
during  instruction,  the  primary  function  of  cues  is  to  direct  students’  attention  and  reduce  the  amount  of  time 
necessary  to  acquire  the  desired  information  which  is  to  be  learned.  He  continued  to  state  that  cueing  strategies  have 
been  employed  to  improve  students  learning  and  these  strategies  has  been  identified  as  “being  effective  in  attracting 
and  sustaining  student  attention  for  extended  period  of  time  . . . which  have  been  very  effective  in  focusing,  isolating, 
and  structuring  information  being  transmitted  (p.  159). 

F.  M.  Dwyer  suggested  that  there  are  two  types  of  functions  of  instructional  cueing  strategies  (1978).  The 
first  type  of  function  is  to  facilitate  students  to  extract  and  concentrate  on  the  intended  instructional  stimuli  from 
their  overall  perceptual  field;  while  the  second  function  of  cueing  strategies  is  to  provide  elaborated  relevant 
information  or  stimuli  to  help  learners  to  make  complete  understanding  of  the  information  which  they  are 
perceiving.  Gagne  (1985)  proposed  a descriptive  theory  of  strategy  consisting  of  nine  events  of  instruction.  He 
assumes  that  attention  gaining  is  the  initial  event  of  instruction.  There  is  a perception  that  learning  could  be 
enhanced  by  eliminating  many  of  the  extraneous  stimuli  which  may  interfere  with  that  learning  (Dwyer,  1972, 

1978).  Within  the  environment,  a large  amount  of  information  input  into  human  system  in  constant,  due  to  the 
limitations  of  the  capacity  of  human  system  to  process  all  available  stimuli  simultaneously,  “the  perceptual  system 
must  be  highly  selective”  (Dember  & Warm,  1979,  p.  125).  During  this  process,  attention  plays  an  important  role  in 
selecting  key  information  from  the  complex  environment,  so  “ the  concept  of  attention  is  the  focusing  of  awareness” 
(Dember  & Warm,  1979,  p.  125)  and  some  stimuli  are  attended  to  while  other  stimuli  are  disregarded  (Moore, 
Burton,  & Myers,  2003).  This  process  has  been  identified  as  selective  attention  and  defined  as  “the  ability  to  attend 
voluntarily  to  some  attributes  of  the  stimulus  array  and  to  ignore  other  attributes”  (Enns  & Girgus,  1985,  p.  319).  It 
would  be  argued  that  if  cueing  strategies  provide  a good  way  to  gain  the  attention  of  a student  then  cues  would  direct 
a student  to  attend  to  the  most  critical  features  of  a computer-based  instructional  screen  text  display  and  allow 
important  instructional  information  to  be  emphasized.  Meanwhile,  the  attention-gaining  cues  also  can  serve  as  an 
organizational  function  to  facilitate  learners  to  make  relationship  between  ideas  more  apparent.  Researchers  (Taylor, 
Canelos,  Belland,  Dwyer,  & Baker,  1987)  support  this  belief  and  reported  that  attention  indicating  strategies  can 
improve  learning. 

Gagne  (1985)  states  that  elaboration  is  one  of  the  crucial  processes  to  the  encoding  and  retrieving 
processes.  He  addressed  that  elaboration  is  a process  whereby  supporting  information  is  added  to  the  information 
being  learned.  Effective  elaborations  serve  to  link  related  propositions  to  stimulate  retrieval  of  the  learning  context 
(Reder,  1982).  Reder  (1979)  also  supports  the  idea  that  elaborative  encoding  helps  integrate  new  information  with 
prior  knowledge.  He  summarizes  the  list  of  forms  of  elaborations:  addition  of  details  to  information,  clarifying  ideas, 
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explaining  and  contrasting  two  or  more  concepts,  making  inferences,  visualizing  an  image  of  material  to  be  learned, 
making  analogies  to  relate  new  ideas  to  familiar  ideas,  or  associating  new  material  with  previous  knowledge  or 
experience.  Elaborative  processing  often  leads  to  improvements  in  learning  performance  (Aderson  & Reder,  1979). 
Stewart  (1988)  also  identified  several  functions  of  cueing  strategies  that:  (1)  as  an  instructional  strategy,  cueing 
needs  to  enable  and  facilitate  the  readers  to  recognize  the  expected  learning  goals  and  implement  appropriate 
methods  to  finish  the  task;  and  (2)  cueing  needs  to  represent  the  structure  and  organization  of  the  text  and  activate 
the  learning  schema. 

Based  on  the  theoretical  and  empirical  literature  review,  cueing  strategies  are  expected  to  help  students  to 
see  relationships  between  and  among  facts,  concepts,  and  procedures  and  facilitate  higher-order  information 
processing  to  increase  achievement. 

Theoretical  Background  of  Cueing  Strategies 

In  order  to  understand  the  functions  of  cueing  strategies  in  instructional  process,  it  is  important  to  be  aware 
of  its  cognitive  theoretical  background-information  processing  theory.  Cognitive  information  process  is  a branch  of 
cognitive  psychology  that  considers  how  people  perceive,  process,  and  act  on  information  and  focus  on  attention, 
perception,  and  memory  (Ausubel,  1968).  Cognitive  psychologists  have  offered  numerous  explanations  about  how 
people  mentally  process  information.  Gredler  (2001)  addresses  that  information-processing  theory  discusses  the 
“basic  steps  in  the  way  individuals  obtain,  code,  and  remember  information,  is  the  central  theory  of  the  cognitive 
perspective”  (p.  169).  The  nature  of  cognitive  process  can  be  extremely  helpful  to  instructional  designers  not  only 
to  understand  what  they  want  students  to  learn,  but  also  how  learners  can  be  effectively  learn  it  (Ormrod,  2003). 


Information  Processing  Theory  and  Cueing  Strategies 

The  origin  of  information-processing  models  can  be  traced  to  Atkinson  and  Shiffrin’s  (1968)  model  of 
memory,  which  was  the  first  suggests  that  memory  consisted  of  a sensory  register,  a short-term  and  long-term  store. 
This  multistage  model  presents  a single,  long-term  store  for  human  memories  (Gredler,  2001).  According  to 
Atkinson  and  Shiffrin,  learners  are  assumed  to  receive  information  from  the  environment  and  then  transform  it  for 
storage  for  future  retrieve.  Sensory  memory  helps  learners  to  organize  the  information  that  they  received  from  the 
environment  and  begin  the  process  of  recognizing  and  coding  the  information.  Since  information  can  only  exist  in 
the  sensory  memory  for  an  extremely  brief  period  about  0. 5-2.0  seconds  (Gredler,  2001),  learners  must  choose  to 
attend  to  the  information  if  it  is  to  continue  on  for  future  processing.  Figure  1 demonstrates  the  multistage  cognitive 
information  processing  (Driscoll,  2000). 


Cognitive  Information  Processing 


Figure  1 Cognitive  Information  Processing 

Short-term  memory  or  working  memory  is  the  stage  in  which  attention  has  been  paid  and  permits  learners 
to  store  information  temporally  in  memory  so  as  to  make  connection  with  previous  information.  Working  memory  is 
where  much  of  the  thinking  or  cognitive  processing  occurs  (Ormrod,  2003).  In  this  stage,  much  of  the  information  is 
encoded  into  some  meaningful  form  and  transferred  into  long-term  memory.  Unfortunately,  short-term  memory  is 
limited  in  its  capacity  and  duration  (Gillani,  2003).  People  only  can  attended  to  a small  amount  of  information  at  any 
one  time  which  means  that  attention  has  a limited  capacity.  In  the  period  of  moving  information  to  working 
memory,  since  of  the  limited  capacity  of  human  attention,  only  a very  small  amount  of  information  in  one’s  sensory 
register  can  move  on  to  working  memory  (Ormrod,  2003).  Gradler  (2001)  emphasizes  that  information  in  short-term 
memory  only  can  be  maintained  for  about  20  seconds  without  rehearsed.  Obviously,  it  is  critical  that  students  need 
to  pay  attention  to  the  things  that  they  are  supposed  to  learn.  Wilshire  (1989)  states  that  in  the  human  information 
processing  system,  the  major  weakness  is  the  short-term  memory.  He  mentions  that  in  order  to  design  an  effective 
instruction,  recognizing  the  limitations  of  short-term  memory  and  the  need  for  providing  appropriate  retrieve  cues  is 
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very  important.  G.  A.  Miller  (1956)  presents  the  idea  that  short-term  memory  could  only  hold  five  to  nine  chucks  of 
information  (seven  plus  or  minus  two),  where  a chunk  is  any  meaningful  unit.  The  unit  of  chunk  could  be  digits, 
words,  chess  positions,  or  people’s  faces.  The  concept  of  chunking  and  the  limited  capacity  of  short-term  memory 
became  a basic  element  of  all  subsequent  theories  of  memory  (Owens,  2002). 

Long-term  memory  is  the  last  step  in  information  processing  which  makes  learners  remember  and  apply 
information  long  after  it  has  been  originally  learned.  Klatzky  (1975)  described  long-term  memory  as  “storehouse” 

(p.  12)  with  unlimited  capacity  for  storing  information.  She  mentioned  two  important  hypotheses  about  long-term 
memory  which  are  worth  noting.  The  first  one  is  that  long-term  memory  is  a permanent  store,  although  sometimes 
people  cannot  remember  everything.  The  forgetting  phenomenon  has  been  described  as  a retrieve  problem  that  the 
information  is  there,  but  people  cannot  get  it.  Driscoll  (2000)  reports  that  the  process  of  retrieve  can  be  greatly  affect 
by  the  cues  available  to  learners  at  the  test  that  “whatever  cues  are  used  by  a learner  to  facilitate  encoding  will  also 
serve  as  the  best  retrieve  cues  for  that  information  at  test  time”  (p.  103).  Another  hypothesis  by  Klatzky  is  that 
“many  different  types  of  information  about  items  must  reside  there  . . . the  variety  of  information  that  can  represent 
an  item”  (p.  13).  Klatzky  (1980)  defined  long-term  memory  as  a mental  dictionary  of  concepts  and  their  associations 
to  each  other.  A characteristic  of  information  that  is  stored  in  long-term  memory  is  that  it  must  have  meaning  and  it 
must  be  integrated  with  related  prior  knowledge.  She  suggests  that  the  more  meaningful  the  information,  the  more  it 
likely  to  be  remembered.  R.  M.  Gagne  (1985)  believes  that  information  stored  in  long-term  memory  is  organized  as 
schemas.  A schema  is  an  organized  body  of  knowledge  about  a specific  topic  which  helps  the  learner  to  relate  pieces 
of  information  to  existing  information.  They  are  influential  in  how  new  information  is  interpreted  and  stored  in  long- 
term memory.  Information  in  long-term  memory  is  believed  to  be  stored  indefinitely  (Rumelhart  & Ortony,  1977). 

Gredler  (2001)  emphasizes  that  there  are  several  essential  components  of  learning  which  are:  (a)  the 
organization  of  information  to  be  learned;  (b)  the  learner’s  prior  knowledge  of  the  instructional  contents;  and  (c)  the 
process  involved  in  perceiving,  comprehending,  and  storing  information.  Be  aware  of  these  essential  components  of 
learning  can  assist  instructional  designers  in  their  design  process  so  as  to  increase  learning  performance.  Norman 
(1969)  states  “the  capacity  of  the  human  to  deal  with  incoming  information  is  severely  limited.  It  is  as  if  at  some 
stage  of  the  analysis  of  incoming  information  only  a small  portion  of  the  incoming  signal  is  selected  for  further 
processing”  (p.  4).  In  a summary  of  the  instructional  value  of  the  immediate  effects  of  attention-getting  devices,  he 
concluded  that  these  devices  are  to  make  the  learners:  (1)  perceive,  (2)  conceive,  (3)  distinguish,  (4)  remember,  and 
(5)  shorten  reaction  time.  It  is  clear  that  strategies  to  focus  student  attention  and  psychological  aspect  of  visual  and 
verbal  information  processing,  storage,  and  recall  are  intertwined  and  extremely  important  to  assist  learning. 


Cognitive  Processes 

In  order  to  understand  the  cognitive  reason  for  using  cueing  strategies,  it  is  important  to  analyze  the 
cognitive  processes  which  supports  the  use  of  cueing  strategies.  Cognitive  processes  have  been  identified  as 
perception,  attention,  encoding,  storage,  and  retrieval  (Labant  & Dwyer,  2002).  M.  L.  Fleming  (1987)  emphasizes 
that  perception,  attention,  and  learning  are  intertwined  together  practically  and  theoretically. 

Perception.  Ausubel  (1968)  defined  perception  as  an  awareness  of  an  object  or  even  prior  to  the  cognitive 
processing  of  that  object  or  event.  Perception  is  a selective  process  (Ausubel,  1968;  Burbank  & Pett,  1989;  M. 
Fleming  & Levie,  1978).  The  amount  of  information  to  which  people  received  is  unlimited,  but  the  total  information 
processing  capacity  is  limited.  Perception  must  be  selective  since  people  only  can  attend  to  a limited  amount  of 
information  at  once  (M.  Fleming  & Levie,  1978).  Ausubel  (1968)  states  that  this  selection  process  is  based  on 
individuals’  experiences,  expectations,  goals,  and  their  individual  differences  that  people  scan  the  available 
information  and  select  to  certain  part  of  the  information  for  processing.  In  order  to  limit  the  amount  of  information 
presented,  instructional  designers  should  use  only  pertinent  information  and  visual  or  verbal  cues  to  focus  attention 
on  key  points  (Burbank  & Pett,  1989).  Burbank  and  Pett  continually  recommend  that: 

Perception  is  organized,  so  arrange  material  in  keeping  with  the  subject  matter.  Use  arrows,  numbers  or 
other  cues  to  direct  the  learner’s  perceptions.  Perception  is  influenced  by  expectations,  so  use  appropriate  formats 
and  cues  such  as  headings,  or  to  aid  learner  in  knowledge  what  to  expect,  (p.  529) 

Alessi  and  Trollip  (1991)  point  out  that  perception  may  be  facilitated  by  presentation  design  factors  such 
as:  “detail  and  realism,  the  use  of  sound  versus  visuals,  color,  characteristics  of  text  such  as  its  size  and  font, 
animation,  and  position  of  screen  elements  . . .”  (p.  11). 

Attention.  This  literature  focuses  discussions  on  attention  value  of  a cue.  Attention  is  an  important  cognitive 
process  which  allows  learners  to  selectively  address  particular  stimuli  for  successful  learning  (Rieber,  1994).  All 
human  learning  is  depending  on  the  learners  to  attend  to  stimuli  and  correctly  perceiving  them  (Alessi  & Trollip, 
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1991).  Learners  are  selectively  focus  on  information  they  think  is  important  and  ignore  information  they  considered 
unimportant. 

Attention  in  learning  has  been  long  recognized  by  cognitive  researchers  (Driscoll,  2000).  Norman  (1969)  in 
discussion  of  the  principles  of  memory  and  attention  in  human  information  processing,  noted  that  the  effects  of 
attention  are  perceive,  conceive,  distinguish,  remember,  and  shortens  ‘reaction  time  ’ . Information  must  first  be 
noticed  in  order  to  be  remembered  in  long-term  memory.  So  to  pertain  to  their  learning  goals,  learners  selectively 
attend  to  certain  stimuli.  An  important  aspect  of  instruction  is  in  structuring  the  learning  environment  so  that  the 
learner’s  attention  can  be  focused  (Bruning,  Schraw,  & Ronning,  1995). 

Researchers  provided  several  theoretical  models  of  attention  which  explain  how  information  be  focused  on 
in  information  processing.  Broadbent’s  (1958)  filter  model  of  attention  describes  how  learners  select  a message  to 
process  information.  It  proposed  that  selective  attention  acts  like  a filter  to  block  some  information  and  lets  only 
certain  types  of  information  pass  to  the  processing  unit.  Filter  model  asserts  that  attention  was  a limited  capacity 
channel  that  determined  the  serial  processing  of  the  perceptual  system  (Broadbent,  1958).  Between  the  large 
capacity  sensory  memory  and  the  further  processing  unit,  there  is  only  one  single  channel  that  transmits  information. 
So  there  is  only  a small  amount  of  available  information  is  selected  for  the  further  processing.  Broadbent  stresses 
that  a particular  message  can  be  selected  and  attended  to  on  its  physical  basis.  However,  later  studies  have  indicated 
that  attention  as  based  solely  on  physical  characteristics  of  the  stimulus  is  not  true  (Klatzky,  1975;  Treisman,  1964). 
Klatzky  (1975)  indicates  that  learners  might  select  messages  based  on  semantic  content  also.  Treisman  (1964) 
argues  that  the  learners  would  attenuate  and  not  filter  out  certain  message  based  on  their  physical  properties.  She 
proposed  that  attention  more  like  the  attenuator,  which  turns  down  the  volume  of  the  unattended  channels  instead  of 
blocking  them  out.  She  explained  that  all  incoming  signals  are  go  through  a preliminary  test  which  not  only  examine 
their  physical  characteristics,  but  also  analyze  their  contents.  The  test  determines  which  information  will  be  attended 
to  and  which  one  is  not.  After  this  preliminary  test,  attention  would  be  directed  to  one  of  the  channels.  In  1973, 
Shiffrin  and  Geisler  (cited  in  Bourne  Jr.,  Domino wski,  Loftus,  & Healy,  1986)  presented  a unlimited  capacity 
model.  They  argued  that  the  attentional  “bottleneck”  is  only  a memory  phenomenon  and  it  does  not  exist  prior  to 
short-term  memory.  They  proposed  that: 

There  is  no  limit  on  our  capacity  to  recognize  stimuli  coming  from  different  channels  at  the  same  time,  and 

the  process  of  recognizing  a stimulus  arriving  on  one  channel  does  not  disturb  the  process  of  recognizing  a 

second  stimulus  arriving  at  the  same  time  on  another  channel,  (p.  61) 

Reiber  (1994)  addresses  that  there  are  two  kinds  of  attention.  One  is  made  subconsciously,  such  as  sensory  input,  the 
other  is  made  consciously  such  as  selective  attention  which  refers  to  people  focus  on  one  set  of  information  while 
ignore  or  block  other  incoming  information  on  purpose.  Kahneman  (1973)  introduced  another  model  of  attention 
which  support  Rieber’s  opinion  of  attention  that  in  addition  to  unconscious  processes,  attention  can  be  consciously 
focused.  The  model  initiates  the  idea  that  attention  can  be  view  as  a learning  strategy  which  can  be  improved. 

Researchers  (R.  C.  Anderson,  1982;  Van  Dijk  & Kintsch,  1983)  have  proposed  a selective  attention  theory 
which  indicated  that  important  text  elements  are  better  learned  and  recalled  because  extra  attention  is  allocated  to 
them.  The  following  is  the  explanation  of  the  selective  attention  theory  by  R.  C.  Anderson  (1982): 

(1)  Text  elements  are  processed  to  some  minimal  level  and  graded  for  importance. 

(2)  Extra  attention  is  devoted  to  elements  in  proportion  to  their  importance. 

(3)  Because  of  the  extra  attention,  or  a process  supported  by  the  extra  attention,  important  text  elements 
are  learned  better  than  other  elements,  (p.  292) 

The  theory  predicts  “a  positive  relationship  between  text  element  importance,  attention  focused  on  those 
elements,  and  eventual  learning”  (Reynolds  & Shirey,  1988,  p.  82).  Reynolds  and  Shirey  also  address  that  inserting 
cues  in  the  text  can  make  particular  text  elements  become  important. 

Driscoll  (2000)  described  attention  as  a state,  a resource,  or  a process.  Attention  as  a state  occurs  “when  a 
learner  maintain  an  attitude  of  expectation,  alert  to  information  and  heedless  of  distractions”  (p.  276).  Those  features 
exist  in  learners  who  are  interested  in  what  they  are  learning.  For  students  who  are  not  interested  in  the  instruction  or 
suffer  from  the  incapability  of  focusing  on  attentions  are  easily  distracted  from  a learning  task.  “Attention  as  a 
resource  refers  to  a learner’ s capability  of  selectively  allocating  more  attention  to  one  of  several  simultaneously 
occurring  events  . . . attention  as  a process  involves  selecting  particular  information  for  future  analysis  and 
interpretation  over  other,  available  information”  (Driscoll,  2000,  p.  277).  Learner’s  ability  of  attention  is  a cognitive 
development  process.  Young  learners  are  easily  to  focus  on  attention  on  one  thing  to  another.  Meanwhile,  they  are 
also  easily  drawn  their  attention  to  irrelevant  objects  and  events.  But  as  young  learners  become  older,  their  ability  to 
focus  attention  on  a particular  task  and  less  likely  to  be  distracted  by  irrelevant  occurrences  becomes  better 
(Dempster  & Corkill,  1999). 
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Gaining  attention  has  been  listed  as  the  first  of  the  nine  events  of  a successful  instruction  (R.  M.  Gagne, 
1985).  Knowledge  cannot  be  obtained  unless  the  learner  is  in  some  way  oriented  to  incoming  information.  Keller’s 
(cited  in  Driscoll,  2000)  model  of  motivational  design  suggests  the  first  process  of  instruction  is  gaining  attention  of 
learning  and  engaging  them  in  the  learning  activity  before  any  instruction  and  learning  occur.  Klatzky  (1975)  states 
that  when  the  incredible  amount  of  information  impinges  on  the  sensory  memory,  some  of  the  information  is 
important  and  some  is  not.  Since  memory  system  cannot  recognize  all  of  the  information  at  once  and  pass  it  to  the 
short-term  memory,  attention  acts  to  filter  out  the  less  important  information  and  select  the  more  important  ones  for 
future  processing.  McKeachie  (1988)  states  that  learning  strategies,  aimed  at  helping  learners’  attention  process,  can 
influence  how  much  attention  is  paid  and  how  much  information  reach  short-term  memory  for  future  process. 

All  of  these  models  and  theories  explained  attention  in  different  views  and  provided  valuable  guidelines  to 
help  practitioners  to  have  a good  understanding  on  some  phenomena  of  attention  in  information  processing.  In 
addition,  Norman  (1969)  also  asserts  that  “one  thing  that  must  of  certainty  be  true  of  the  study  of  attention  is  that  it 
cannot  be  divorced  from  the  study  of  other  cognitive  phenomena”  (p.  37). 

Encoding.  The  process  of  encoding  occurs  in  short-term  or  working  memory  which  is  the  step  to  transfer 
the  information  from  short-term  memory  into  long-term  memory.  Driscoll  (2000)  describes  encoding  as  a process 
which  relates  incoming  information  to  the  previous  existed  concepts  or  knowledge  in  order  to  make  the  new 
information  easily  memorable  and  meaningful.  Mayer  (1988)  suggests,  “learning  strategies  aimed  at  the  process  of 
encoding  process  can  influence  how  fast  information  is  encoded  and  how  much  is  encoded”  (p.  16).  In  other  words, 
these  strategies  are  important  because  the  manner  in  which  information  is  treated  in  short-term  memory  determines 
its  value  at  a later  time.  Hodes  (1994)  lists  three  categories  of  encoding  strategies:  repetition,  organization,  and 
elaboration.  Repetition  refers  to  rote  memorization  where  information  is  simply  repeated.  Organization  refers  to 
organizing  information  into  meaningful  units  for  increased  retention.  Hodes  noted  that  elaboration  could  be  verbal 
and  nonverbal  where  learners  construct  statement  or  mental  images  about  the  underlying  meaning  of  the 
information. 

Storage.  Storage  is  the  process  by  which  new  information  is  integrated  in  various  ways  with  previous 
known  information.  Chunking  has  been  viewed  as  an  effective  strategy  to  increase  short-term  memory  storage 
capacity  (Klatzky,  1975).  The  function  of  long-term  memory  is  to  store  information  for  later  use  (E.  D.  Gagne, 
Yekovich,  & Yekovich,  1993).  Long-term  memory  storage  includes  rehearsal,  meaningful  learning,  elaboration, 
organization,  and  visual  knowledge  (Omrod,  1998).  Frase  (1975)  found  the  benefit  of  elaborative  processing  with 
text  materials  for  long-term  storage.  Frase  stresses  that  subjects  who  had  been  given  advanced  organizer  as  an 
elaborative  process  did  better  to  test  questions  than  subjects  in  control  group  without  advance  organizer.  Learners’ 
memory  for  text  materials  would  be  improved  by  elaborative  processing  which  include  strategies,  such  as 
previewing,  questioning,  reading,  reflecting,  reciting,  and  reviewing  (Thomas  & Robinson,  1972).  Furthermore, 
researchers  (Pressley,  McDaniel,  Turnure,  Wood,  & Ahmad,  1987)  address  that  the  most  critical  feature  of 
elaborations  is  whether  the  elaboration  can  focus  learners’  attention  on  the  appropriate  instructional  material  for 
encoding  into  long-term  memory.  In  addition,  learners  learn  and  remember  new  information  more  readily  when  the 
information  is  organized.  Organization  involves  making  connections  between  the  parts  of  the  new  information  in 
order  to  make  the  learning  more  relevant  and  improve  the  long-term  memory  storage  (Gredler,  2001). 

Retrieval.  Mayer  (1988)  describes  retrieval  as  the  process  of  transferring  knowledge  from  long-term 
memory  to  short-term  memory.  The  process  of  retrieval  in  long-term  memory  can  be  described  as  an  indexlike 
mechanism  (Bourne  Jr.  et  al.,  1986).  Making  multiple  connections  with  the  learner’s  existing  knowledge  allows  for 
better  retrieval  of  newly  learned  information  (Ormrod,  1995).  Researchers  (J.  R.  Anderson,  1995;  Ormrod,  1995) 
listed  some  reasons  of  the  failure  to  locate  information  in  long-term  memory  leads  to  retrieval  failure:  (a)  lack  of  a 
relevant  retrieval  cue,  (b)  lack  of  multiple  connections  between  various  knowledge  segments  in  long-term  memory, 
(c)  Infrequent  practice  or  infrequent  use  of  stored  information  can  also  lead  to  retrieval  failure,  (d)  interfering 
material,  and  (e)  memory  decay.  People  reply  on  cues  to  retrieve  information  from  long-term  memory  (Bourne  Jr.  et 
al.,  1986).  Tulving  and  Thompson  (1973)  demonstrate  that  when  providing  the  right  retrieval  cues,  learners  can 
perform  better  recalls  in  their  learning  process.  Tulving  (1974)  stresses  that  memory  must  be  viewed  as  a joint 
function  of  stored  information  and  the  information  provided  to  the  subject  at  retrieval,  such  as  retrieval  cues. 
Continually,  Tulving  (1983)  indicates  that  retrieval  cues  can  be  viewed  as  the  particularly  salient  or  significant 
“aspects  of  the  individual’s  physical  and  cognitive  environment  that  initiate  and  influence  the  process  of  retrieval” 

(p.  171).  For  a given  retrieval  cues  to  be  effective,  it  must  be  encoded  with  the  original  event  and  have  some  sort  of 
connections  with  the  target  item  which  it  is  suppose  to  cue  (Bourne  Jr.  et  al.,  1986). 

Mayer  (1982;  1984)  addresses  three  primary  functions  of  cognitive  process  which  are:  (1)  to  guide  selective 
attention  towards  certain  information  in  the  text;  (2)  to  foster  the  establishing  of  internal  connections  among  ideas 
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from  the  text;  and  (3)  to  foster  building  external  connections  between  ideas  in  the  text  and  learners’  existing 
previous  knowledge. 

Information  processing  theory  provides  the  theoretical  base  for  understanding  the  effects  of  cueing 
strategies  in  learning.  By  analyzing  these  relationships,  instructional  designers  will  get  a better  understanding  on 
employing  effective  instructional  strategies  to  accommodate  student’s  needs  so  as  to  enhance  learning. 

Cueing  Strategies  and  Academic  Achievement 

Various  cueing  strategies  have  been  identified  in  the  literature  as  methods  for  focusing  attention  which 
include  auditory  cue/music  or  narrator,  pictures,  animation,  diagram,  arrow,  underline,  labeling,  color  coding,  text 
size  or  fonts,  motion,  shading,  texture,  questions,  concept  maps,  and  advanced  organizer,  etc.  (Burton,  Moore,  & 
Holmes,  1995;  Dwyer,  1978;  Higgins  & Cocks,  1999;  Levie  & Lentz,  1982).  Most  studies  on  the  effect  of  cueing 
strategies  to  academic  achievement  were  conducted  to  examine  the  effectiveness  of  directing  attention  to  the  specific 
information  with  visuals  in  designing  instructional  message  (Downs,  1989;  Downs  & Jenkins,  2001;  McIntyre, 

1981).  The  research  yielded  mixed  results.  Most  of  the  studies  had  positive  results  (Beck,  1987;  Dinnel  & Glover, 
1985;  Dwyer  & Moore,  1992;  Goldstein,  1981;  Higgins  & Cocks,  1999;  Levie  & Lentz,  1982;  Martel,  1992;  Mayer, 
1989;  McIntyre,  1981).  However,  some  also  had  negative  or  non- significant  research  findings  (Owens,  2002; 
Pienkowski,  2002;  Worley  & Moore,  2001). 

Although  a great  deal  of  research  explored  the  effects  of  cueing  strategies  that  complement  visual  imagery 
in  facilitating  student  achievement,  the  effectiveness  of  cueing  strategies  in  computer  screen  delivered  text  message 
has  not  been  widely  investigated.  Future  research  needs  to  be  conducted  to  clarify  this  issue. 


Instructional  Message  Design 

Researchers  have  begun  to  identify  the  elements  of  the  message  which  are  important  in  facilitating  learning 
achievement  during  instructional  message  design  process  (Lamberski  & Mayers,  1980).  Grabowski  (1991)  described 
message  design  is  a process  of  “planning  for  the  manipulation  of  the  physical  form  of  the  message”  (p.  206).  Here, 
the  message  works  as  a medium  to  communicate  a sender  and  a receiver.  Studies  of  message  design  have  suggested 
that  the  message  should  be  clearly  defined  and  set  forth  to  provide  optimum  instructional  effectiveness  and 
efficiency  in  the  learning  process  (Lamberski  & Mayers,  1980).  In  order  to  guarantee  the  efficient  learning  to  occur, 
“. . . the  stimulus  material  for  learning  must  be  available  and  clearly  perceivable  by  the  learner”  (Dwyer,  1978,  p. 
158).  Fleming  and  Levie  (1978)  have  defined  instructional  message  design  as  “the  process  of  manipulating,  or 
planning  for  the  manipulation  of,  a pattern  of  signs  and  symbols  that  may  provide  the  conditions  for  learning  (p.  xi). 
Instructional  design  must  find  a way  to  magnify  or  emphasize  the  critical  features  in  the  instructional  message  (M. 
Fleming  & Levie,  1978).  Previous  researchers  (Ausubel,  1968;  Rothkopf,  1966)  found  that  positive  learning  was 
affected  by  using  strategies  to  direct  the  learners  attention  to  relevant  information.  Fleming  and  Levie’ s definition 
and  explanation  of  instructional  message  design  “limited  messages  to  the  pattern  of  signs  or  symbols  that  modifies 
the  cognitive,  affective  or  psychomotor  behavior”  (Seel  & Richey,  1994,  p.  31).  According  to  the  interests  of  this 
article,  instructional  message  design  can  be  defined  as  the  process  of  manipulating  the  learning  conditions  that 
encompasses  the  modification  of  the  pattern  of  a signs  or  symbols  (words  or  pictures)  which  has  been  viewed  as  the 
media  to  communicate  learning.  Usually  this  process  will  be  conducted  by  the  teachers  or  instructional  designer 
before  the  instruction. 

Researchers  (Heines,  1984;  Soulier,  1988)  addressed  that  about  80  to  90  percent  of  the  information  was 
presented  by  text  in  computer-based  instruction.  Although  with  the  development  of  computer  technologies  and 
integration  of  multimedia  into  the  curriculum,  instructional  message  can  be  delivered  by  using  graphic  and 
animation  as  well  as  text,  the  majority  of  instructional  materials  in  computer-based  instructional  learning 
environment  are  still  rely  on  text.  This  emphasizes  the  important  effects  of  the  computer  screen  delivered  text 
message  design  on  students’  academic  performance. 

How  information  is  constructed  by  the  reader  to  gain  comprehension  from  text  materials  is  an  important 
goal  for  teaching  and  learning  (Bennett,  1989).  Text  design  has  been  identified  as  one  of  the  factors  which  influence 
learners’  comprehension  of  the  learning  information.  McConaughy  (1985)  suggests  that  one  of  the  essential  aspects 
of  text  comprehension  is  the  ability  to  attend  and  focus  on  important  or  critical  relevant  materials  within  the  text  and 
to  exclude  the  nonessential  materials.  Bennett  (1989)  follows  the  same  reasoning  and  states  that  information 
comprehension  process  is  an  interactive  and  assimilative  process  between  the  learner  and  text  which  depends  upon 
the  cues  used  by  the  learner  to  gain  meaning  from  the  text.  In  order  to  increase  greater  comprehension,  instructional 
designer  should  look  at  the  text  design  and  learners  differences.  R.  C.  Anderson  (1982)  concluded  that  text  message 
should  be  designed  to  allow  readers  to:  assimilate  textual  information,  facilitate  selective  allocation  of  attention, 
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enable  inferential  elaboration,  allow  for  orderly  searches  of  memory,  facilitate  editing  and  summarizing,  and  permit 
inferential  reconstruction. 

Well  designed  text  can  be  used  to  help  learners  to  organize  and  locate  information  and  facilitate 
information  to  be  transformed  and  comprehended  during  information  processing  (Hannafin  & Hooper,  1993). 
Burbank  and  Pett  (1989)  provides  a number  of  principles  and  guidelines  for  designing  text  message  and  instructional 
content.  They  suggests  that  effective  instructional  design  should  include:  “1)  use  variations  in  size,  color,  brightness, 
and  shape  or  employ  novel  visuals  to  attract  attention;  and  2)  to  hold  attention,  be  certain  that  the  material  presented 
is  meaningful  to  the  learners  ”(p.  528).  According  to  Burtank  and  Pett’s  (1989)  suggestions  on  text  design, 
instructional  designers  can  improve  students’  learning  achievement  by:  1)  using  a list  or  outline  of  the  content  to 
assist  learners  to  organize  information;  2)  using  bullets,  numbers  or  white  space  to  separate  and  attend  on  key  items; 
3)  using  red  or  blue  color  to  attract  attention;  and  4)  using  color  to  provide  structure,  or  emphasize  relevant  cues  and 
to  distinguish  critical  items  from  irrelevant  ones.  They  also  state  that  when  developing  textual  materials,  it  is  better 
to  consider  the  characteristics  of  the  audience,  the  information  to  be  presented,  and  the  way  the  audience  will 
process  the  information  (Burbank  & Pett,  1989). 

The  question  of  “how  text  can  be  effective  presented  on  computer  screen”  requires  professionals  to  look  at 
the  text  attributes  in  depth.  Meanwhile,  by  analyzing  and  modifying  text  attributes,  instructional  designers  should  be 
able  to  find  effective  instructional  strategies  to  enhance  readers’  comprehension  since  learning  from  text  message 
not  only  just  pass  reading,  it  requires  learners’  active  text  processing  (Duchastel,  1982).  According  to  online 
information  which  was  retrieved  from  the  Monron  City  Schools’  website  (2005),  text  attributes  refer  to  font,  size, 
style,  and  color.  Those  attributes  of  text  can  be  manipulated  in  a computer-based  environment  within  an 
instructional  message.  Meyer  (1975)  has  identified  that  in  the  text  to  emphasize  how  ideas  are  related  and  which 
ideas  are  most  important,  text  structured  can  be  manipulated  to  influence  free  recall  performance  by  inserting  cues, 
or  signals.  Signaling  or  cueing  content  and  structure  has  been  demonstrated  to  facilitate  low  or  average  learners’ 
recall  performance  (B.  J.  F.  Meyer,  Brandt,  & Bluth,  1980).  Pace  (1982)  summarized  a general  principle  for  the 
design  of  text  that  text  message  designer  should  facilitate  readers  to  acquire  new  information  by  highlighting 
important  ideas  and  reduce  the  density  of  the  information  load.  The  explicit  techniques  which  designers  used  to 
capture  and  focus  attention  during  reading  include  “linguistic,  spatial,  and  typographic  cues  to  the  form,  function, 
sequence,  content,  and  importance  of  segments  of  a passage”  (Jonassen  & Kirschner,  1982,  p.  123).  During  this 
design  process,  several  aspects  need  to  be  considered:  the  nature  of  the  target  audience,  such  as  their 
cognitive/learning  style,  the  expected  outcomes  of  the  text  message,  and  certain  text  characteristics  affect  perception 
of  the  information  (Jonassen  & Kirschner,  1982). 


Conclusion 

To  design  effective  learning  materials,  the  instructional  designers  need  to  ask  those  questions:  How  does 
the  human  mind  work?  How  do  people  process  information?  And  how  can  we  adapt  instructional  strategies  to  aid 
human  cognition?  In  approaching  the  design  of  instructional  text  message,  the  instructional  designers  have  to  have  a 
clear  understanding  of  the  process  of  how  people  learn  and  what  are  the  variables  which  can  affect  this  process. 
They  should  provide  learners  with  stylistically  appropriate  cues  to  organize  the  content  and  direct  learner’ s attention 
so  as  to  improve  learning  performance. 
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Abstract 

Some  may  think  that  a summer  online  course  is  just  a condensed  version  of  a regular  semester  online  course.  If 
the  instructor  compresses  a 15 -week  course  content  into  a five-week  timeframe  and  the  students  triple  their 
time  and  efforts  studying  online,  the  results  of  learning  should  he  the  same.  In  fact,  due  to  the  time  constraint, 
different  instructional  strategies  should  he  used  for  the  design  and  delivery  of  the  short-  and  long-term  online 
courses  in  order  to  retain  the  quality  of  learning  without  reducing  the  amount  of  instruction.  This  paper 
presents  a blended  approach  combing  constructivist  and  objectivist  instructional  strategies  for  the  design  of  an 
intensive  summer  online  course  based  on  a support-oriented  model.  The  results  of  course  evaluation  revealed 
that  students  had  a very  positive  learning  experience  with  the  course  and  were  highly  satisfied  with  their 
learning  outcomes. 


Introduction 

With  the  widespread  concepts  of  life-long  learning  and  just-in-time  training,  many  academic  institutions 
are  rushing  headlong  into  the  e-learning  arena  and  are  offering  a variety  of  online  programs  and  courses  not  only 
during  regular  semesters  but  also  during  summer  sessions  to  meet  the  needs  of  students. 

Some  may  think  that  a summer  online  course  is  just  a condensed  version  of  a regular  semester  online 
course.  It  works  the  same  way  as  face-to-face  classroom  instruction.  If  the  instructor  squeezes  a 15-week  course 
content  into  a five-week  time  frame  and  the  students  triple  their  time  and  efforts  studying  online,  the  results  of 
learning  should  be  the  same.  In  fact,  due  to  the  nature  of  online  communication,  the  asynchronous  mode  in 
particular,  both  teachers  and  students  usually  invest  much  more  time  in  a course  taught  online  than  in  the  same 
course  taught  in  a traditional  classroom  (Palloff  & Pratt,  1999).  Thus,  compressing  the  course  content  and  extending 
the  study  hours  may  not  be  an  effective  approach  for  teaching  and  learning  an  intensive  online  course.  How  to 
retain  the  quality  without  sacrificing  the  quantity  of  learning  is  a challenge  to  instructional  designers  and  instructors 
of  intensive  online  courses.  This  paper  presents  a blended  approach  combing  constructivist  and  objectivist 
instructional  strategies  for  the  design  of  an  intensive  summer  online  course  based  on  a support-oriented  model. 


Constructivist  vs.  Objectivist  instruction 

Over  the  past  two  decades,  instructional  design  paradigms  have  shifted  from  objectivism  to  constructivism 
(Bonk  & Cunningham,  1998;  Cooney,  1998;  Vrasidas,  2000;  Tam,  2000).  During  the  same  time,  the  development  of 
Internet  technology  has  expanded  learning  environments  from  traditional  classrooms  to  online  and  changed  the 
nature  of  teaching  and  learning  experiences.  This  change  is  leading  instructional  designers  to  adopt  constructivist 
learning  theories  to  the  design  of  online  learning  environments. 

The  positive  value  of  employing  constructivist  theories  in  the  design  of  online  learning  environments  has 
been  supported  by  educational  literature  (Hannafin,  Land,  & Oliver,  1999;  Moallem,  2003;  Morrison;  2003;  Palloff 
& Pratt,  1999,  2001).  Cognitive  constructivists  believe  that  knowledge  is  constructed  by  the  learner  through  an 
internal  process  of  interpreting  the  world  based  on  the  learner’s  prior  knowledge  and  experience  (Jonassen,  1992). 
Social  constructivists  view  learning,  rather  than  as  a purely  internal  cognitive  processing  of  information,  as  a social 
construct  of  knowledge  via  interaction  which  is  mediated  by  language  or  social-cultural  dialogue  (Bonk  & 
Cunningham,  1998;  Dewey,  1938;  Vygostsky,  1978).  This  school  of  theorists  believes  that  the  context  in  which  the 
learning  occurs  is  foremost  to  the  learning  itself  (McMahon,  1997);  thus  instruction  should  involve  authentic 
learning  tasks  replicating  real-world  context  and  activities  to  optimize  knowledge  application  (Jonassen,  1999; 
Savery  & Duffy,  1996). 

Jonassen  (2004)  concluded  that  learning,  in  order  to  be  meaningful,  must  involve  authentic  learning  tasks 
requiring  cognitive  information  processing  and  knowledge  construction  within  the  context  of  collaboration  and 
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social  interactions.  Harasim  (1990)  noted  that  the  online  environment  is  particularly  appropriate  for  collaborative 
learning  because  it  emphasizes  group  interaction.  Through  group  collaboration  and  interaction,  social  isolation  can 
be  reduced  if  not  prevented  (Hughes,  Wickersham,  Ryan-Jones,  & Smith,  2002).  Active,  authentic,  practical,  and 
meaningful  are  some  of  the  advantages  of  constructivist-based  instruction. 

In  contrast  to  constructivists,  objectivists  believe  that  knowledge  exists  independent  of  the  learner. 
Learning  means  a change  in  the  learner’s  behavior  or  cognitive  structure  (Lakoff,  1987)  and  learning  is  an 
independent  effort  of  individuals  rather  than  collaboration  and  interaction  among  peers.  Thus,  objectivist-based 
instruction  emphasizes  the  appearance  of  observable  and  measurable  behavior  in  individual  students. 

Objectivist  instructional  design  theories,  derived  from  behaviorist  and  cognitive  psychology,  advocate 
systematic  planning  of  instructional  materials  and  learning  activities  so  that  students  can  follow  the  prescribed  steps 
to  obtain  the  desired  learning  outcomes.  With  detailed  and  well- structured  instruction  and  feedback  based  on 
dichotomous  right/wrong  judgment  of  the  student’s  response,  the  objectivist  instructional  approach  has  been 
traditionally  used  for  the  design  of  classroom  and  televised  instruction  and  computer-based  individualized  learning. 
This  approach  grants  to  instructors  the  control  of  components  of  learning  such  as  students,  environment,  materials, 
activities,  and  time.  Efficiency  and  predictability  are  part  of  the  strengths  of  objectivist-based  instruction.  The 
following  table  compares  the  characteristics  of  constructivist  and  objectivist  instruction: 


Table  1:  Comparison  of  characteristics  of  constructivist  and  objectivist  instruction 


Constructivist  instruction 

Objectivist  instruction 

Student-centered 

Instructor-centered 

111- structured 

Well- structured 

Less  detailed 

More  detailed 

Subscribed  (active  learning) 

Prescribed  (passive  learning) 

Contextual 

Out  of  context 

Multiple  perspectives 

Single  perspective 

Collaborative  interdependent  learning 

Individualized  independent  learning 

Less  efficient 

More  efficient 

Goal-free  evaluation 

Goal-based  evaluation 

There  is  an  increased  interest  among  teachers  and  instructional  designers  in  adopting  the  constructivist 
philosophy  of  teaching  and  learning  in  the  design  and  delivery  of  online  courses.  However,  in  a constructivist 
learning  environment,  in  order  to  successfully  promote  active  and  meaningful  learning,  the  instructor  as  a learning 
facilitator  has  to  coach,  scaffold  and  monitor  students’  knowledge  construction;  provide  immediate  feedback;  form 
and  norm  groups;  encourage  collaboration,  interaction,  and  multiple  perspectives;  and  assist  in  learning  reflection 
and  interpretation.  These  activities  require  a great  commitment  of  time  and  energy  from  the  instructor.  Students 
engaged  in  constructivist  learning  also  have  to  spend  a lot  of  time  and  effort  not  only  on  learning  and  interaction  but 
also  on  managing  logistical  tasks  such  as  coordinating  with  other  group  members.  Compared  to  objectivist-based 
instruction,  constructivist-based  teaching  and  learning  require  more  time  and  effort  from  both  the  instructor  and  the 
students  (Cavanaugh,  2005;  Hughes,  Wickersham,  Ryan-Jones,  & Smith,  2002;  Rajandran,  2003). 

There  would  less  likely  be  a problem  if  a constructivist-based  online  course  were  offered  during  the  regular 
semester.  Students  have  more  time  to  work  collaboratively  with  their  peers  to  construct  knowledge  and  to  solve 
assigned  problems  through  synchronous  and  asynchronous  communication.  However,  it  would  be  a challenge  for 
the  instructor  to  implement  the  same  constructivist-based  instruction  in  an  intensive  summer  course. 

If  time  is  the  essential  factor  affecting  the  successful  completion  of  a constructivist-based  intensive  online 
course,  instructional  designers  may  consider  using  a blended  approach,  which  combines  the  strengths  of 
constructivist  and  objectivist  methods  of  teaching  and  learning  for  the  design  of  the  course.  Thus,  meaningful 
learning  can  be  achieved  efficiently. 


Conceptual  framework 

Technology  and  support 

Online  learning  requires  a certain  level  of  technological  knowledge  and  skill.  Inadequate  technological  skill 
and  technical  difficulties  not  only  can  result  in  anxiety,  frustration,  confusion  and  disorganization  for  the  student  but 
it  can  also  impede  the  communication  and  interaction  process,  thus  hindering  group  collaboration  (Ge,  Yamashiro, 

& Lee,  2000;  Ragoonaden  & Bordelrau,  2000).  The  impact  of  technology  problems  is  more  serious  and  urgent  in  an 
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intensive  than  a regular  semester  online  course  because  the  short  time  span  of  the  intensive  course  leaves  little  room 
for  technical  problem-solving  and  skill  fostering.  To  efficiently  develop  students’  competence  and  comfort  with 
technology,  designing  an  easily  accessed,  user-friendly  technical  support  system  should  be  the  foremost 
consideration  for  the  design  of  an  intensive  online  course. 

Learning  and  supports 

As  noted  by  the  constructivist  theories,  meaningful  learning  must  be  student-centered  and  involve  authentic 
learning  tasks  requiring  cognitive  information  processing  and  knowledge  construction  via  collaboration  and  social 
interaction  (Jonassen,  2004).  Jonassen  and  Howland  (2004)  also  indicated  that  providing  the  learners  with 
meaningful  and  consequential  tasks  can  help  form  and  foster  learning  communities  (p.72). 

Since  the  nature  and  complexity  of  a task  has  a direct  influence  on  group  collaboration  and  communication, 
selecting  a learning  task  is  one  of  the  most  critical  considerations  for  online  course  design.  Constructivist-oriented 
tasks  have  the  potential  to  promote  meaningful  learning,  interpersonal  and  teamwork  skills,  and  higher  level 
thinking  skills,  but  they  demand  more  time  for  learning  and  collaboration.  On  the  other  hand,  objectivist-oriented 
tasks  require  less  time  and  social  support  and  thus  can  be  achieved  more  efficiently  but  the  learning  is  less 
meaningful. 

To  maximize  successful  online  learning,  a learning  support  system  offering  rich  resources,  guidance  and 
assistance  from  the  instructor,  experts,  and  peers  should  be  provided.  In  addition,  a social  support  system  must  be 
made  available  so  that  members  of  the  learning  community  trust  each  other  and  feel  comfortable  in  sharing 
knowledge,  feelings,  experiences,  values  and  goals. 

In  all,  there  are  three  major  components  involved  in  online  learning:  technology,  content,  people 
(instructor,  students,  experts,  etc.).  Students  (Ss)  construct  knowledge  by  interacting  with  people  and  content 
through  the  use  of  technology  to  complete  the  learning  tasks.  To  optimize  learning  outcomes,  each  component  must 
be  sustained  by  it  own  support  system  - technical  support  system  for  technology,  learning  support  system  for 
content,  and  social  support  system  for  people.  The  selection  of  instructional  design  strategies  should  reflect  a 
balanced  consideration  between  time  constraints  and  meaningful  learning.  With  these  thoughts  in  mind,  a support- 
oriented  learning  environment  was  conceptualized  (see  Figure  1)  to  guide  the  design  of  an  intensive  online  course. 


Figure  1 : Conceptual  framework  for  support-oriented  online  learning  environment 

About  the  course 

The  course  was  a three-credit  100%  online  course  for  graduate  education  majors  regarding  the  integration 
of  technology  in  the  K-12  curriculum.  The  course  was  originally  offered  as  a regular  semester  online  course  by  a 
university  in  the  north-central  region  of  the  United  States.  However,  to  meet  the  demand  for  the  course,  the  course 
was  redesigned  to  fit  into  a five- week  summer  session  schedule  which  was  then  offered  in  both  short  summer 
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sessions  in  addition  to  the  regular  semester  offering.  The  course  was  housed  in  an  online  course  management  system 
(CMS)  developed  by  the  university.  The  CMS  offered  course  design  and  communication  tools  and  online  technical 
support  resources. 


Course  design  specifications 

The  summer  course  design  was  based  on  the  same  instructional  goals  for  the  regular  semester  course. 
However,  to  adapt  to  the  shorter  time  frame,  the  course  redesign  used  a blended  approach  combing  constructivist 
and  objectivist  views  of  learning  for  the  design  of  learning  tasks,  activities,  and  the  three  support  systems  (technical, 
learning,  and  social)  with  an  aim  at  meaningful  and  efficient  learning  in  a supportive  environment. 

Instructional  design  principles  based  on  the  blended  approach 

❖ Meaningful  learning  - Adopting  the  constructivist  approach,  this  course  used  authentic  problem-based  learning 
tasks  associated  with  practical  context  so  that  the  learning  experience  can  be  applied  to  the  real  world. 

❖ Efficient  learning  - Applying  the  objectivist  approach,  the  course  design  increased  scaffolding  by  providing 
more  clear  and  detailed  guidance  and  instruction  and  assigning  group- supported  individual  projects  rather  than 
group  projects  to  accommodate  time-limit. 

❖ Supportive  learning  - Instructional  strategies  based  on  constructivist  and  objectivist  approaches  were  employed 
for  the  design  of  technical,  learning  and  social  supports  so  that  students  would  feel  comfortable  with  the 
technology,  content,  instructor  and  fellow  students. 

> Technical  support  - 

■ Providing  easily-accessed  user-friendly  technology  resources  such  as  job  aid,  FAQ,  university 
technology  helpdesk  contact  information,  and  CMS  tech  support  site. 

■ Providing  efficient  technical  training. 

■ Creating  an  “S.O.S.”  forum  for  the  class  members  to  ask  and  receive  help  from  the  instructor  and  other 
students  regardless  of  technical  or  learning  issues. 

> Learning  support 

■ Selecting  meaningful,  practical  and  feasible  tasks  requiring  a reasonable  amount  of  cognitive 
challenge. 

■ U sing  group-  supported  proj  ec ts . 

■ Providing  rich  instructional  resources  such  as  worked  examples,  FAQ,  Internet  resources,  online 
library,  online  experts,  online  and  offline  instructor  office  hours. 

■ Providing  more  structured  instructional  materials  and  instructional  guidance  whenever  needed. 

> Social  support 

■ Forming  a collaborative  online  learning  community  using  the  following  strategies: 

Informing  students  of  the  value  of  collaboration. 

Assigning  grade  to  team  contribution. 

Helping  develop  group  norm,  i.e.  appropriate  netiquette,  online  communication,  value  of 
collaboration,  and  group  role  and  responsibility. 

Embedding  collaboration  and  interaction  in  group- supported  projects. 

Encouraging  collaboration  and  interaction  among  group  members  to  gain  multiple  perspectives 
and  solutions. 

Providing  opportunities  for  social  presence  and  interaction. 

Pre-instructional  (Orientation) 

Given  the  time  constraint  of  an  intensive  course,  preparing  the  students  for  online  learning  prior  to  formal 
instruction  is  critical  and  has  a direct  impact  on  the  success  of  online  learning,  especially  for  those  who  are  new  to 
online  learning.  A course  orientation  was  offered  to  build  students’  readiness  for  online  learning  (Hughes  et  al. 
2002).  The  strategies  used  for  the  design  of  course  orientation  included  the  following: 

• provide  meaningful  and  efficient  technical  training 

• provide  an  easily-accessed  user-friendly  technical  support  resources 

• embed  orientation  assignments  in  technical  training 

• provide  knowledge  about  collaboration  and  interaction 

• familiarize  students  with  the  course,  instructor,  and  students 

• initiate  learning  community  building 


146 


Comfort  and  trust  with  technology  (an  example  of  technical  support  system) 

In  addition  to  the  online  course  management  system  (CMS)  support  site  provided  by  the  university,  the 
instructor  employed  objectivist  instructional  strategies  to  design  the  following  easily-accessed  and  user-friendly 
resources  to  be  included  in  the  technical  support  system: 

- A step-by-step  “how-to”  job  aid  for  each  of  the  tools  required  by  the  course. 

- A customized  online  technical  training  module  focusing  only  on  the  communication  and  production  tools 
needed  by  the  course.  This  saved  students’  time  from  completing  the  entire  CMS  tutorial. 

- A pre-instruction  survey  regarding  their  technical  skill  and  professional  background. 


Comfort  and  trust  with  instructor  and  other  students  (an  example  of  social  support  system) 

As  part  of  the  course  orientation  activities,  students  were  asked  to  study  the  information  on  the  orientation 
page  regarding  how  to  collaborate  and  interact  with  each  other  using  appropriate  netiquette.  In  order  to  get 
acquainted  with  each  other,  students  were  required  to  post  a self-introduction  message  to  the  “Hello!”  social  forum 
and  to  respond  to  their  peers’  postings  and  to  upload  their  digital  photo  to  their  designated  student  profile  page.  This 
activity  initiated  the  social  presence  and  interaction  of  the  class  members  as  a learning  community. 

Comfort  with  the  course  (an  example  of  learning  support  system) 

Providing  students  with  course  expectations  and  content  map  can  help  organize  their  study  plan.  Informing 
students  of  the  required  prerequisite  will  help  connect  their  existing  cognitive  structure  with  the  new  content 
information  (Mohamed,  2004).  This  information  was  included  in  both  the  orientation  and  the  syllabus  pages. 
Students  were  required  to  read  this  information  to  get  familiar  with  the  course.  Students  were  also  asked  to  send  the 
instructor  their  expectations  of  the  course  via  CMS  e-mail  attachment  to  let  the  instructor  know  students’  needs. 

Instruction 

Learning  tasks 

McGrath’s  Task  Circumplex  (1984)  classifies  group  tasks  into  four  categories,  each  containing  two  task 
types:  generate  (type  1 and  2),  choose  (type  3 and  4),  negotiate  (type  5 and  6),  and  execute  (type  7 and  8). 

Group  and  individual  time  spent  on  task  and  learning  activities  should  be  determined  and  considered  when 
designing  an  online  course  (Barrows,  1996;  Savery  & Duffy,  1995).  Taking  the  time  constraint  and  the  goals  of  the 
course  into  account,  three  types  of  McGrath’s  tasks  were  adapted  for  the  design  of  the  learning  tasks:  Type  1- 
Planning  Tasks  (generate  action-oriented  plans),  Type  2-  Creativity  Tasks  (generate  creative  ideas),  and  Type  3- 
Intellective  Tasks  (choose  correct  answers  to  solve  problem).  Although  all  three  task  types  are  suitable  for  group 
collaborative  projects,  task  type  1 and  2 were  used  for  group- supported  individual  projects  while  type  3 was  used  for 
individual  projects. 

The  constructivist  instructional  principles  used  to  guide  the  design  of  generative  tasks  (types  1 and  2) 
include  anchoring  all  learning  activities  to  a larger  task,  using  authentic  tasks  in  a real-world  setting  with  meaningful 
context,  using  complex  tasks  requiring  thinking  and  judging,  and  allowing  multiple  perspectives  and  solutions 
(Savery  & Duffy,  1996).  For  the  intellective  tasks,  objectivist  instructional  principles  were  used  to  prescribe  clear 
and  detailed  instruction  and  assessment  based  on  well-defined  goals  and  objectives.  The  culminating  task  was  a 
group- supported  project  requiring  an  integration  of  skills  learned  from  all  previous  units  which  consisted  of  both 
generative  and  intellective  tasks. 

An  example  of  a group- supported  individual  project  would  be  creating  a course  plan  integrating  Internet 
and  computer  technology  into  the  instruction  of  the  course  for  a subject  area  and  grade  level  of  the  students’  choice. 
Students  were  asked  to  use  the  school  where  they  were  employed  as  the  problem  context  if  applicable;  otherwise,  a 
case  scenario  simulating  a real-life  situation  would  be  supplied  by  the  instructor.  There  were  only  appropriate  but  no 
standard  correct  answers.  In  order  to  create  a meaningful  and  effective  course  plan,  students  had  to  know  how  to 
conduct  an  effective  web  search  for  appropriate  resources  to  be  used  in  the  plan,  devise  an  instrument  to  evaluate  the 
quality  of  web  resources,  write  the  evaluation  report,  and  develop  a technology  integration  plan.  An  example  of  an 
individual  project  would  be  writing  a website  evaluation  report  which  was  based  on  the  result  of  an  absolute  rating 
scale  and  some  narrative  comments. 

Learning  activities 

Forming  groups  is  one  of  the  most  important  activities  related  to  the  success  of  a collaborative  problem- 
solving experience  (Bridges,  1992;  Slavin,  1995  in  Nelson,  p.  259).  Small  groups  of  at  least  three  and  no  more  than 
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six  have  been  shown  to  be  the  most  effective  size  (Bruffee,  1993;  Johnson  & Johnson,  1997;  Putnam,  1997). 
Heterogeneous  grouping  promotes  diversity,  multiple  perspectives,  interpersonal  skill,  and  team  work  (Nelson, 
1999). 

To  avoid  wasting  time  on  coordinating  group  members  and  their  share  of  work,  group- supported  individual  projects 
were  assigned  instead  of  group  projects.  To  accomplish  the  group- supported  assignments,  a learning  community  of 
the  class  was  organized  by  project  groups,  each  consisting  of  three  to  four  students  with  different  backgrounds 
(whenever  possible)  assigned  by  the  instructor  based  on  the  results  of  the  pre-instruction  survey. 

Group  members  were  required  to  contribute  ideas  during  project  brainstorming  discussion,  and  then  each 
student  worked  on  his/her  own  individual  projects.  In  addition  to  instructor’s  review,  students  were  also  required  to 
review  and  comment  on  other  group  members’  draft  projects  to  help  improve  the  quality  of  the  work  and  to  learn 
from  each  other  before  final  submission  of  the  projects.  Students  were  made  aware  that  their  contribution  to  the 
group  would  be  graded.  With  such  an  arrangement,  students  were  working  on  their  own  projects  while  still 
contributing  to  the  group  and  interacting  with  other  group  members  and  the  instructor.  The  learning  community 
remained  active  and  supportive. 

Post-instruction  (Assessment) 

The  assessment  of  students’  learning  product  was  based  on  the  objectivist  goal-based  evaluation.  Clear  and 
detailed  assignment  specifications,  including  purpose  of  the  assignment,  submission  procedure,  due  date,  and  rubric 
for  grading  criteria  were  provided.  A constructivist  approach,  requiring  the  students  to  write  a final  reflection  paper, 
was  used  for  learning  process  assessment. 


Course  evaluation 

At  the  end  of  the  course  implementation,  a summative  evaluation  was  conducted  to  answer  the  following 
questions: 

1.  What  were  students’  feelings,  attitudes  and  opinions  about  their  learning  experience  of  this  course? 

2.  How  did  the  course  designs  influence  students’  process  and  product  of  learning? 

Data  and  analysis 

■ Students’  response  to  the  course  evaluation  administered  by  the  university  (11  participations  with  returning  rate 
100%). 

■ Students’ exit  reflections 

■ Students’ project  results 

The  course  evaluation  surveyed  students  feelings,  attitudes,  and  opinions  about  their  learning  experience  of 
this  course  using  38  items  of  5-point  rating  scale  (4-Strongly  agree,  3-Agree,  2-Undecided,  1 -Disagree,  0-Strongly 
disagree)  and  7 open-ended  items.  1 1 students  enrolled  the  course  and  the  return  rate  for  the  course  evaluation  is 
100%. 

A descriptive  data  analysis  procedure  was  performed  to  analyze  students’  average  response  rates  to  the 
course  evaluation.  The  content  of  the  exit  reflections  and  open-ended  items  in  the  course  evaluation  were  analyzed 
qualitatively  using  content  analysis  based  on  the  categories  of  most  and  least  valuable  aspects  of  the  course,  most 
and  least  liked  course  activities,  and  the  overall  learning  experience  with  the  course. 

Results 

The  quantitative  analysis  results  of  the  closed-ended  items  in  the  course  evaluation  are  reported  in  table  2. 


Table  2:  Results  of  students’  feelings,  attitudes  and  opinions  about  course  learning  experience 


Item  description 

Mean 

SD 

Feelings  about  the  quality  of  the  course 

3.60 

.06 

Feelings  about  the  learning  experience 

3.87 

.06 

Opinions  about  the  design  of  learning  tasks,  activities  & instruction 

3.68 

.26 

Opinions  about  the  instructional  methods  used  by  the  instructor 

3.73 

.20 

Opinions  about  the  value  of  personal  and  professional  development 

3.6 

.06 

Attitudes  about  group- supported  collaborative  learning 

3.64 

.13 
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The  content  analysis  results  are  classified  into  the  categories  of  most  and  least  valuable  aspect  of  the 
course,  most  and  least  liked  learning  activities,  and  overall  learning  experiences  of  the  course.  Table  3 reports  the 
analysis  results  together  with  the  excerpts. 


Table  3:  Results  of  content  analysis  of  course  evaluation  and  exit  reflections 


Categories 

Excerpts 

Most  valuable  aspects  of  the  course: 

1.  Authentic  projects 

2.  Instructor’s  feedback 

3.  User-friendly  technology 

“the  final  project.” 

“the  rubrics.” 

“Kinesthetic  experience  with  technology  and 
Internet  projects.” 

“Instructor  feedback  on  course  project.” 

“The  instructor  was  very  helpful  in  holding  us  to 
where  she  wanted  us  to  be!” 

“The  simple  use  of  the  computer  and  the  Internet  to 
accomplish  what  was  needed  for  this  class.” 

Least  valuable  aspects  of  the  course: 
1.  Readings 

“The  readings  to  me  were  not  very  valuable.” 
“Too  much  lecture  readings.” 

Most  liked  class  activities: 

1.  Working  on  class  projects 

2.  Working  with  group 

3.  Working  with  diverse  students 

4.  Online  discussion  and  chat 

“I  liked  the  projects  we  did.” 

“I  liked  learning  the  process  for  designing  Internet 
lesson.” 

“I  enjoyed  taking  classes  with  people  from  all  over 
the  world.” 

“I  enjoyed  working  with  a wide  variety  of  students 
from  different  regions  within  the  USA  and  even 
different  countries.” 

“I  also  appreciated  the  diversity  of  the  classmates.” 
“I  really  enjoyed  the  interaction  of  the  students 
about  our  projects.  It  helped  me  to  refine  my 
thoughts.” 

“I  enjoyed  the  online  chat  with  my  group 
members.” 

“I  liked  how  groups  were  set  up  to  help  each  other 
and  that  groups  could  arrange  their  own  meeting 
time  to  do  online  chat  discussions  that  suited 
EVERYONE'S  needs.” 

Least  liked  class  activities: 

1.  Working  with  CMS 

2.  Developing  rubrics 

“I  did  not  like  the  fact  that  we  could  not  access  the 
announcements  during  assignments,  or  to  upload  a 
file  was  had  to  jump  completely  out  and  then  get 
back  in.” 

“I  HATE  doing  rubrics.” 

Overall  learning  experience  with  this  online  course: 

1.  Great 

2.  Wonderful 

3.  Enjoyed 

4.  Fruitful 

“The  course  was  great  overall.” 

“My  learning  experience  was  wonderful.  I really 
learned  a lot  through  actually  doing  and 
experiencing  the  things  we  were  learning  about.” 

“I  learned  something  each  lesson  and  did  not  feel 
my  time  was  wasted.  I will  use  this  stuff  at  work.” 
“It  was  a good  experience  for  me  and  helped  me  to 
think  outside  the  usual  box  for  learning  something 
new.” 

“I  learned  a great  deal  about  evaluating  Web  sites.  I 
also  learned  how  to  make  up  my  own  Internet-based 
program  which  was  very  interesting.” 

“I  am  very  happy  with  this  course.” 

“I  enjoyed  the  course  work  and  the  development  of 
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new  items  and  plans.” 


Discussion  and  Conclusion 

Students’  ratings  on  the  course  evaluations  showed  that  they  are  highly  satisfied  with  the  course  as  well  as 
their  learning.  The  content  analysis  results  of  course  evaluation  and  exit  reflections  revealed  that  students  in  general 
valued  the  authentic  projects,  instructor’s  feedback,  and  user-friendly  technology  the  most.  Although  a couple  of 
students  commented  on  the  amount  and  the  necessity  of  the  readings  in  the  lecture,  majority  of  the  students  thought 
the  readings  aided  to  their  understanding  of  the  subject  and  built  knowledge  foundation  for  project  assignments. 

The  class  activities  that  the  students  liked  the  best  were  working  on  class  projects,  working  with  groups, 
working  with  diverse  students,  and  online  discussion  and  chat.  One  student  disliked  the  access  procedure  of  the 
CMS  and  the  other  student  did  not  enjoy  the  development  of  rubrics  for  the  lesson  plan.  The  overall  learning 
experience  with  this  course  was  very  positive  across  the  entire  class.  They  enjoyed  the  learning  experience  and  felt 
they  learned  a lot  within  the  short-time  frame  without  feeling  the  pressure. 

The  quality  of  students’  product  for  each  unit  was  assessed  against  a rubric  in  which  grading  criteria  were 
clearly  specified.  This  scaffolding  strategy  helped  guide  the  students’  attention  to  the  important  aspects  of  the 
product.  Eight  of  eleven  (73%)  students  received  an  A and  three  (27%)  received  an  A minus  for  the  course  grade. 

In  conclusion,  the  evaluation  results  support  the  use  of  a blended  approach  combing  objectivist  and 
constructivist  instructional  strategies  for  the  design  of  an  online  course  when  it  is  pressed  by  time.  The  following 
guidelines  may  help  design  effective  intensive  online  courses: 

1.  Ensure  students’  comforts  with  technology,  content,  and  people  by  providing  technical,  learning,  and 
social  supports. 

2.  Use  the  orientation  activities  to  prepare  for  students’  readiness  for  online  learning  in  terms  of 
technology,  instructional  content,  and  collaboration. 

3.  Balance  between  the  individual  and  group  collaborative  activities. 

4.  Provide  learning  tasks  that  are  meaningful  and  manageable  in  a given  time  frame. 

5.  Provide  clear  and  concise  instructional  content  and  assignment  specifications. 

6.  Provide  timely  feedback  and  more  scaffolds  to  assist  learning. 
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Abstract 

This  paper  designs  an  intelligent  tutoring  system  for  individualizing  instruction  based  on  learning  style. 
The  system  builds  on  theories  from  individual  differences,  case-based  reasoning,  and  learning  objects.  The 
prototype  is  designed  to  give  various  learners  having  different  learning  styles  personalized  instruction  which  is 
constructed  with  learning  objects  in  terms  of  meta-data.  Case-based  reasoning  will  be  used  to  diagnose  learners' 
differences  and  analyze  the  relationship  between  learners'  achievement  and  preference.  The  design  is  the  first  step 
to  implement  the  system. 


Introduction 

Adaptive  learning  systems  as  an  extension  of  web-based  instruction  have  been  studied  by  many  researchers 
(e.g.,  Magoulas,  Papanikolaou  & Grigoriadou,  2003;  Yacci,  2000  ).  It  aims  to  help  learners  gain  their  own 
knowledge  structure  by  personalizing  instruction  based  on  individual  differences.  Individual  differences  are 
important  to  the  design  of  web-based  instruction  because  all  learners  do  not  use  web-based  instruction  effectively 
(Chen,  2003).  In  previous  studies,  the  construction  of  individualized  instruction  appropriate  to  the  learner’s 
characteristics  heavily  relied  on  a predetermined  design  by  the  researcher.  Throughout  these  studies,  the  instruction 
available  to  the  students  was  static.  What  has  yet  to  be  researched  is  whether  instruction  can  evolve  over  time  to  be  a 
closer  match  to  the  individual’s  difference. 


Theoretical  framework 

This  system  is  built  on  a combination  of  three  theories:  individual  differences,  learning  objects,  and  case- 
based  reasoning  (CBR).  Many  studies  have  been  conducted  to  identify  the  relationship  between  learning  success  and 
individual  differences,  such  as  learning  style,  cognitive  style,  and  prior  knowledge  (e.g.,  Hsu  & Dwyer,  2004;  Foster 
& Lin,  2003).  Individual  differences  have  had  significant  statistical  effects  on  student  learning  in  web-based 
instruction  (Hsu  & Dwyer,  2004;  Rurke  & Lysynchuk,  2000).  In  contrast,  other  studies  reveal  that  the  web  has  been 
proven  to  be  an  equally  effective  learning  environment  for  students  regardless  of  individual  differences  (Aragon, 
Johnson,  & Shaik,  2000;  Shih  & Gamon,  2002). 

To  deliver  appropriate  personalized  instruction,  an  instructional  unit  should  be  subdivided  so  that  various 
levels  of  difficulty  and  sequences  can  be  constructed.  The  IEEE  Learning  Technology  Standard  Committee  (LTSC) 
defines  a learning  object  as  “any  entity,  digital  or  non-digital  that  can  be  used,  re-used,  or  referenced  during 
technology  supported  learning”  (LTSC,  2000).  Learning  objects  are  the  smallest  units  within  a larger  instructional 
structure  (Wiley,  2000),  and  they  can  be  sequenced  and  combined  as  the  main  components  of  instruction.  In 
addition,  metadata  is  a resource  description  about  content,  quality  or  features  of  learning  objects  (Dillon,  2000). 
Metadata  offers  the  cataloguing  information  necessary  to  share,  use  and  reuse  learning  objects.  Therefore,  metadata 
provides  the  mechanisms  necessary  to  match  learning  objects  to  an  individual’s  characteristics. 

One  method  of  building  instruction  using  learning  objects  relies  on  CBR.  CBR  is  based  on  human  intuition 
in  which  new  problems  are  often  similar  to  previously  encountered  problems;  therefore,  past  solutions  may  be  used 
in  the  current  situation.  In  2002,  Mamaghani  described  the  process  of  CBR  comprising  four  Re’s:  (1)  retrieve  the 
most  similar  case,  (2)  reuse  the  case  to  attempt  to  solve  the  problem,  (3)  revise  the  proposed  solution  if  necessary, 
and  (4)  retain  the  solution  as  part  of  a new  case.  CBR  has  been  applied  to  educational  realms  as  an  answering  system 
in  distance  learning  (Tsinakos,  2003),  problem-solving  emulation  (Sormo  & Admodt,  2002),  and  the  Knowledge 
Innovation  for  Technology  in  Education  (KITE)  project  (Moore,  Means  & Kim,  2004).  The  case-based  reasoner  can 
play  an  important  role  in  making  instruction  individualized  because  it  can  make  effective  decisions  based  on 
previous  cases. 


Prototype  of  the  system 

The  proposed  prototype  of  Matching  Learning  Object  to  Individual  Differences  (MLOID)  system  focuses 
on  delivering  the  most  effective  instruction  by  using  case-based  reasoning  (CBR)  based  on  an  accumulated  database. 
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The  instruction  in  the  system  consists  of  learning  objects  associated  with  metadata.  The  MLOID  system  is  designed 
to  determine  a learning  style  of  a learner  and  offer  appropriate  learning  objects  to  a student  in  accordance  with  his 
learning  preference  and  prior  knowledge  using  case-based  reasoning.  All  learning  processes  and  achievement  will  be 
recorded  as  cases  according  to  each  learner  and  learning  style.  The  case-based  reasoner  will  use  the  cases  as  a 
reference  to  make  appropriate  instruction  when  a previous  learner  returns  to  study  but  also  when  a new  learner 
begins  to  study. 

The  MLOID  system  consists  of  diagnosis,  the  case-based  reasoner,  the  learning  model  database,  the 
learning  objects  database,  customized  instruction,  and  assessments  as  depicted  in  figure  1.  The  diagnosis  of  learning 
style  for  MLOID  determines  one  of  four  characteristics  of  learning  styles:  imaginative,  dynamic,  analytic,  or 
common  sense  based  on  McCarthy’s  (1990)  4MAT  system.  Another  diagnosis  determines  the  learner’s  prior 
knowledge.  The  case-based  reasoner  refers  to  cases  in  the  learning  model  database.  The  learning  model  database 
accumulates  cases  of  students’  profiles  and  previous  customized  instruction.  The  reasoner  determines  an 
instructional  unit  which  is  applicable  to  the  student’s  learning  style  and  prior  knowledge.  After  an  appropriate  unit  is 
selected,  learning  objects  are  retrieved  as  necessary  to  generate  the  customized  instruction.  Finally,  as  assessments 
are  completed  throughout  the  instructional  unit,  the  student’ s profile  is  revised  and  retained  in  the  learning  model 
database. 


MLOID 


Student 


Inventory 


Customized  instruction 


Reuse 


Diagnosis 


Assessment 


Customizing 


Learning 

Objects 


Case-Based  Reasoner 


Retrieve 


Revise 


Retain 


Learning  Model  Database 

(Accumulated  cases) 


- The  relationships  among  learning  style,  prior 
knowledge,  and  learning  objects 

- Each  student’s  profile 


[Figure  1]  The  architecture  of  the  MLOID  system 


In  addition  to  the  instructional  content,  the  aim  of  the  MLOID  is  to  offer  a student  appropriate  instruction 
with  a customized  interface  and  navigation  comprised  of  learning  objects  so  it  will  maximize  learning  achievement 
for  each  individual. 

In  order  to  focus  on  the  individualization,  the  system  must  have  three  basic  algorithms  in  the  case-based 
reasoner:  matching  a standard  of  learning  style  to  a characteristic  of  various  learning  objects,  selecting  relevant 
levels  of  learning  objects  according  to  the  learner’s  prior  knowledge,  and  developing  dynamic  instructional  web 
pages  comprised  of  learning  objects  based  on  the  previous  two  analyses.  When  a student  first  logs  in  to  the  system, 
the  student  is  asked  to  answer  an  inventory  which  diagnoses  a learning  style  and  prior  knowledge.  The  case-based 
reasoner  analyzes  the  diagnoses  for  congruence  with  previous  cases,  and  the  system  selects  necessary  learning 
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objects.  The  learner  receives  their  customized  instructional  unit  with  customized  interface  and  navigation  comprised 
of  the  learning  objects. 

In  a basic  CBR  process,  the  system  retrieves  a successful  learning  case  of  learning  objects  that  is  most 
efficient  to  the  characteristics  detected  by  the  diagnosis.  The  high  ranked  case  is  reused  to  make  an  instructional 
unit.  After  conducting  the  unit,  the  case  is  revised  and  retained  by  the  case-based  reasoner  based  on  the  result  of 
assessment.  In  this  step,  the  system  accumulates  not  only  a record  of  the  individual’s  progress  but  also  the 
relationships  among  learning  style,  prior  knowledge,  and  learning  objects  as  a new  case,  adding  to  the  learner’s 
profile.  The  new  profile  is  used  as  a basis  of  reasoning  to  decide  whether  the  same  kind  of  instructional  unit  will  be 
offered  when  the  learner  needs  feedback  or  moves  to  next  step.  In  addition,  the  accumulation  of  cases  will  be  helpful 
in  making  instructional  units  for  other  individuals. 


Future  work 

Case-based  reasoning  can  make  a web-based  instructional  system  evolve  into  effective  decision-making 
system,  referencing  an  accumulated  database.  However,  there  are  considerations  in  developing  the  system.  First,  it 
should  be  thoroughly  analyzed  which  learning  style  model  is  applicable  to  the  MLOID  system.  Second,  CBR 
algorithms  of  matching  learning  objects  should  be  developed.  Third,  an  effective  relational  structure  of  database  is 
needed.  Last,  classifying  the  granularity  of  learning  objects  in  metadata  should  be  considered.  Further  explanation 
about  the  implications  and  limitations  of  implementation  will  be  provided  in  the  presentation. 
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Abstract 

Problem-based  learning  is  increasingly  being  recognized  as  an  effective  approach  for  teaching  science. 
Using  KaAMS  (Kids  as  Airborne  Mission  Scientists)  - a website  that  provides  contextualized  problem-based 
learning  lesson  plans  - this  study  examines  the  interrelationship  between  teacher  characteristics  and  teaching 
practice,  student  characteristics  ( including  problem-solving  skills)  and  the  rate  of  use  of  KaAMS  (the  number  of 
KaAMS  lessons  used  in  a class).  Based  on  an  extensive  review  of  literature,  a model  that  predicts  middle  school 
student  problem-solving  skills  based  on  student  and  teacher  characteristics  and  teaching  practice  is  presented. 


Introduction 

Problem-solving  skills  have  increasingly  been  recognized  as  a desired  outcome  of  learning  that  should  be 
developed  through  K-12  education.  According  to  Mayer  (1989),  “problem-solving  skills  should  be  learned  within 
the  context  of  realistic  problem-solving  situations.  Instead  of  using  drill  and  practice  on  component  skills  in  isolation 
- as  suggested  by  the  skill-based  approach  - a metaskill  based  approach  suggests  modeling  of  how  and  when  to  use 
strategies  in  realistic  academic  tasks”  (p.  53). 

Research  has  established  Problem-Based  Learning  (PBL)  as  a powerful  and  effective  approach  to  science 
teaching.  PBL  has  been  found  to  help  students  develop  deep  understanding  of  content  knowledge  and  improve  their 
problem-solving  skills  by  situating  learners  in  real  world  problem-solving  contexts  (Edens,  2000;  Lovett,  2002; 
Taconis,  Ferguson-Hessler,  & Broekkamp,  2001;  Voss,  1989).  At  the  same  time,  disadvantages  and  limitations  of 
the  PBL  approach  have  been  identified  (Johns,  1996). 

In  the  past  few  years,  the  web  has  become  a popular  medium  for  sharing  PBL  instruction.  However, 
lacking  the  solid  evidence  for  supporting  the  possibility  of  using  the  Internet  as  a medium  for  sharing  PBL 
instruction,  there  is  a growing  need  to  understand  the  impact  of  web-based  PBL  environments  in  facilitating  the 
development  of  problem-solving  skills.  As  a result,  this  study  is  interested  in  the  impact  of  web-enhanced  PBL 
environments  on  student  problem-solving  skills. 

There  is  an  abundance  of  research  in  science  education  that  investigates  students’  problem-solving  skills. 
Several  studies  have  been  conducted  that  investigate  the  impact  of  individual  student  variables  on  student’ s 
problem-solving  skills  - for  example  studies  that  investigate  the  relationship  between  metacognition  and  problem- 
solving skills  (Alexander  & Judy,  1988).  There  are  also  studies  that  investigate  the  relationship  between  individual 
teacher  characteristics  and  student  problem-solving  skills  (Taconis,  et  al,  2001).  Yet,  there  is  currently  no  model  in 
PBL  literature  that  includes  all  of  the  above  mentioned  student  and  teacher  characteristics.  Consequently,  in  order  to 
further  understand  how  different  variables  are  interrelated  with  problem-solving  skills,  this  study  also  aim  to  build  a 
predictive  model  in  PBL  that  elaborates  the  interrelationships  between  various  variables  and  problem-solving  skills. 

Purpose  of  the  Study 

This  study  will  investigate  the  impact  of  a web-enhanced  PBL  curriculum  on  student  problem-solving 
skills.  In  addition,  this  study  also  aim  to  develop  a model  that  relates  middle  school  students’  problem-solving  skills 
with  their  characteristics,  the  characteristics  of  their  teachers,  teaching  practice,  and  the  rate  of  use  of  PBL 
curriculum. 
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A web-enhanced  PBL  curriculum,  KaAMS  (Kids  as  Airborne  Mission  Scientists),  will  be  used  in  this 
study.  It  was  developed,  in  collaboration  with  NASA,  to  facilitate  the  development  of  problem-solving  skills  in 
middle  school  students  and  also  to  inspire  them  to  pursue  careers  in  science,  math,  technology,  and  geography. 

Research  Questions 

The  overarching  research  question  of  this  study  is  “What  is  the  relationship  between  the  rate  of  use  of 
KaAMS,  teacher  characteristics  and  teaching  practice,  and  student  characteristics  including  general 
problem-solving  skills ?” 

More  specifically,  this  research  investigates  the  following: 

1 . Does  KaAMS  have  an  impact  on  teacher  characteristics  (computer  and  Internet  skills  level,  attitude 
toward  the  use  of  web  resources,  teachers’  feeling  of  preparedness  to  use  computers  and  the  Internet, 
pedagogical  beliefs  toward  PBL,  teachers’  science  teaching  efficacy)  and  teaching  practice  (use  of 
general  teaching  strategies,  use  of  Intemet/computer,  and  teachers’  actual  use  of  PBL)? 

2.  Does  KaAMS  have  an  impact  on  student  characteristics  (science  self-efficacy,  self-concept,  science 
career  interest,  science  attitudes,  perceived  usefulness  of  science,  metacognitive,  and  gender)  and 
general  problem-solving  skills? 

Following  is  the  research  model  that  includes  all  the  variables  this  research  will  be  investigating. 
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Research  Methodology 

Our  plan  is  to  conduct  this  research  at  the  national  level.  Middle  school  teachers  and  students  from  all  over 
the  country  will  be  recruited  to  participate  in  this  research.  Teachers  will  be  asked  to  use  KaAMS  curriculum 
materials  in  their  classroom  and  complete  surveys  before  and  after  the  classroom  trials.  Students  will  also  be  asked 
to  complete  student  characteristic  surveys  (again  before  and  after  classroom  use  of  KaAMS). 

This  data  will  be  used  to  develop  a model  that  relates  the  rate  of  use  of  KaAMS  (number  of  KaAMS  lesson 
used  by  a teacher)  with  teacher  characteristics,  teaching  practice,  and  student  characteristics  (including  general 
problem-solving  skills).  Factor  analysis  methods  and  Structural  Equation  Modeling  techniques  will  be  used  to 
analyze  the  data  and  build  a model. 


Review  of  Literature 

Student  Variables 

Several  student  characteristics  that  might  influence  student’s  problem-solving  skills  have  been  identified 
from  relevant  literature.  These  variables  include  science  self-efficacy,  science  self-concept,  science  attitude, 
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perceived  usefulness  of  science,  science  career  interest  and  metacognition.  In  this  session  problem-solving  skills  will 
first  be  introduced  followed  by  definition  of  each  aforementioned  factor  and  their  relationships  with  other  variables. 
In  addition,  a diagram  depicting  how  variables  relate  to  each  other  will  be  presented. 

Performance  on  Problem-solving  Tasks 

In  literature,  performance  on  problem-solving  tasks  is  seen  as  a function  of  both  domain  specific  knowledge 
(knowledge  base:  declarative,  procedural,  or  conditional  knowledge)  and  strategic  knowledge  (problem-solving 
skills  base).  For  example,  a meta  analysis  of  problem-solving  (Alexander  & Judy,  1988)  investigate  the  interaction 
between  domain- specific  knowledge  and  strategic  knowledge.  Based  on  an  extensive  review  of  literature  (research 
that  covers  a range  of  age  groups,  domains  and  methodologies)  they  propose  several  hypothesis  about  this 
interaction:  (a)  domain- specific  knowledge  is  requisite  to  the  effective  utilization  of  strategic  knowledge,  (b) 
strategic  knowledge  is  required  for  acquisition  of  domain  knowledge,  (c)  “ill-formed  or  unintelligent  use  of 
strategies  can  be  detrimental  to  learning”  (p.  390),  and  (d)  “as  knowledge  of  domain  increases  strategic  processing  is 
altered”  (p.  391). 

Problem-solving  Skills 

Problem-solving  skills  are  defined  as  skills  that  are  required  to  successfully  perform  a problem-solving 
task.  These  skills  include  problem  representation,  strategies  of  working  out  a problem,  monitoring  of  the  solution 
process,  as  well  as  the  evaluation  of  the  execution  of  the  problem  (Flavell,  1976) 

Alexander  and  Judy  (1988)  have  called  problem-solving  strategies  a “special  form  of  procedural  knowledge  that  can 
exist  in  varying  degrees  of  generality  or  separation  from  specific  domains.”  They  define  problem-solving  strategies 
as  “goal  directed  procedures  that  are  planfully  or  intentionally  evoked  either  prior  to,  during  or  after  the  performance 
of  a task”  (p.  376).  Voss  (1989)  makes  the  following  observation  about  general  problem-solving  abilities:  “It  has 
been  proposed  that  there  are  general  problem-solving  strategies  (i.e.,  strategies  that  are  applicable  in  a large  number 
of  contexts),  and  that  such  strategies  should  be  taught  in  school.”  (p.  276). 

Dimensions  of  Problem-solving  Skills 

Taconis  and  his  collogues  (2001)  characterize  problem-solving  skills  on  two  dimensions  — general- 
specific  on  the  one  hand  and  simple-complex  on  the  other.  The  authors  provide  the  following  examples: 

a)  General-Specific  dimension:  General  skills  are  skills  that  “cannot  be  exclusively  related  to  one  specific  schema  in 
the  knowledge  base.  . . .General  skills  can  be  related  to  the  general  activities  of  the  problem-solving  strategy,  such  as 
analyzing,  planning,  and  doing  calculations.  Another  group  of  general  skills  required  in  science  problem-solving 
comprises  thinking  skills,  such  as  proportional  reasoning  (Grayson,  1996,  1997)  and  skills  in  building  an  adequate 
mental  representation  out  of  the  schemata  available  (Savelsbergh,  1998).”  (p.  447).  Specific  skills  are  “domain 
specific  and  related  to  procedures  belonging  to  a problem  schema  (Chi,  Feltovich,  & Glaser,  1981;  Taconis,  1995)” 
(p.  447). 

b)  Simple-Complex  dimension:  “Some  skills  are  complex  - for  instance,  building  a representation  of  a given 
situation;  while  others  are  simple  - for  instance,  checking  that  no  information  given  is  forgotten”  (p.  447). 

Metacognition 

The  relationship  between  problem-solving  skills  and  metacognition  has  been  frequently  investigated.  In 
fact  we  note  that  some  researchers  view  problem-solving  skills  as  metacognitive  knowledge.  For  example, 

Alexander  and  Judy  (1988)  consider  met  cognitive  knowledge  to  encompass  knowledge  of  self,  knowledge  of  task, 
and  knowledge  of  strategies.  Flavell  (1976)  defines  metacognition  as  “one’s  knowledge  concerning  one’s  own 
cognitive  processes  and  products  or  anything  related  to  them.  . . .Metacognition  refers,  among  other  things,  to  the 
active  monitoring  and  consequent  regulation  and  orchestration  of  these  processes  in  relation  to  the  cognitive  objects 
or  data  on  which  they  bear. . .”  (p.  232). 

Relationship  between  Problem-solving  Skills  and  Metacognition 

Sternberg  (1998)  suggests  that  metacognition  is  an  important  part  of  the  abilities  that  lead  to  student 
expertise.  For  example,  he  has  written  that  expertise  involves  “(c)  spending  proportionately  more  time  determining 
how  to  represent  test  problems  than  they  do  in  search  for  and  in  executing  a problem  strategy;  (d)  developing 
sophisticated  representations  of  test  problems,  based  on  structural  similarities  among  problems;  . . . (f)  generally 
choosing  a strategy  based  on  elaborate  schemas  for  problem  strategies;  . . . (j)  accurately  predicting  the  difficulty  of 
solving  particular  test  problems;  (k)  carefully  monitoring  their  own  problem-solving  strategies  and  processes;”  (p. 
133).  Sternberg  argues  that  these  aspects  of  expertise  in  problem-solving  directly  involve  metacognition.  (Mayer, 
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1998)  distinguishes  between  skill,  meta-skill  and  will.  He  defines  meta-skill  as  a problem  solver’s  ability  to  control 
and  monitor  cognitive  processes.  Mayer  writes  that  “metacognition  - in  the  form  of  metaskill  - is  central  in 
problem-solving  because  it  manages  and  coordinates  the  other  components”  (p.  53). 

Science  Self-efficacy 

According  to  Bandura  (1986),  self-efficacy  is  a set  of  personal  beliefs  regarding  one’s  capabilities  in 
specific  situations  and  performance  domain.  That  is,  self-efficacy  is  individual’s  beliefs  of  how  well  he  or  she  can 
successfully  execute  actions  required  to  accomplish  some  task  (Bandura,  1977).  These  beliefs  are  referred  to  as  self- 
efficacy  beliefs,  expectations  or  judgments.  In  addition,  Bandura  ’s  self-efficacy  theory  asserts  that  four  major 
sources  of  information  are  influential  in  shaping  and  modifying  self-efficacy  beliefs:  (a)  past  performance 
accomplishments,  (b)  exposure  to  and  identification  with  efficacious  models  (various  learning),  (c)  access  to  verbal 
persuasion  and  support  from  others,  and  (d)  experience  of  emotional  or  physiological  arousal  in  the  context  of  task 
performance.  Past  performance  accomplishments  are  generally  assumed  to  be  most  influential  in  promoting  self- 
efficacy  given  that  they  are  based  on  authentic  mastery  experiences  (Bandura,  1986). 

To  be  more  specific,  Lent,  Brown  and  Larkin  (1986)  found  that  self-efficacy  contributed  significantly  to  the 
prediction  of  science  and  engineering  grades,  persistence  and  range  of  career  options.  Many  studies  are  indicated 
positive  evidences  in  relating  self-efficacy  to  various  variables,  especially  to  student  achievement.  Recently  study  is 
strengthened  previous  assumptions  that  the  efficacy  beliefs  have  a stronger  association  with  academic  performance 
when  compared  to  self  concept  as  well  (Pietsch,  Walker  & Champan,  2003). 

Science  Self-concept 

Self-concept  is  how  people  perceive  themselves.  Super  (1963)  defined  self-concept  as  the  “individual’s 
picture  of  himself.”  This  picture  is  shaped  by  different  components  such  as  the  roles  people  play,  the  situations  they 
are  in  and  the  functions  they  perform.  Many  research  studies  examine  the  relationship  between  self-concept  and 
student  learning,  and  some  of  them  indicated  that  general  self-concept  is  not  significantly  correlated  with  an 
individual’s  achievement  when  it  comes  to  a specific  academic  subject  (Marsh,  Byrne  & Shavelson,  1988). 

To  better  explore  the  relationship  between  self-concept  and  student  learning  in  a specific  academic  subject, 
academic  self-  concept  has  been  developed  (Shavelson,  Hubner,  & Stanton,  1976).  Marsh  (1986)  provided  an 
internal-external  framework  of  reference  model  (I/E  model)  to  explain  how  academic  self-concept  has  been 
developed.  According  to  this  model,  students  form  their  academic  self-concept  by  comparing  their  perceived 
performances  with  those  of  their  classmates  in  an  academic  subject  area  (external  reference).  They  also  compared 
their  own  performances  in  one  subject  with  those  in  other  school  subjects.  For  example,  students  developed  their 
science  self-concept  by  comparing  their  perceived  performances  in  science  with  those  of  their  classmates  in  the 
science  class.  They  also  compared  their  own  perceived  performances  in  since  with  in  other  subject  area,  such  as 
math  and  English. 

In  the  area  of  science  education,  a body  of  research  has  investigated  how  science  self-concept  influences 
student  achievements  as  well  as  other  science-related  variables.  In  a ten-year  longitudinal  study  on  elementary, 
junior  high  and  high  school  students,  Simpson  and  Oliver  (1990)  found  that  science  self-concept  significantly 
predicted  the  number  and  type  of  future  science  courses  the  10th  grade  students  took,  which,  in  turn,  led  to  students’ 
lifelong  interests  and  learning  in  science. 

Science  Attitude 

Science  attitude  is  deemed  as  a factor  that  highly  affects  student  science  learning  and  achievement.  As  a 
result,  science  attitude  has  been  studied  extensively,  but  there  is  no  consensus  among  researchers  on  what  science 
attitude  is  consisted  of.  For  example,  the  instrument  developed  by  Fraser  (1978)  has  been  widely  used  to  investigate 
science  attitude.  This  survey  was  consisted  of  seven  scales:  social  implications  of  science,  normality  of  scientists, 
attitude  toward  scientific  inquiry,  adoption  of  scientific  attitudes,  enjoyment  of  science  lessons,  leisure  interest  in 
science,  and  career  interest  in  science.  To  further  understand  the  concept  of  science  attitude,  Haladyna  and 
Shaughnessy  (1982)  examined  49  relevant  studies  and  identified  six  aspects  associated  with  attitudes  toward  science 
in  these  studies.  These  aspects  included  scientific  attitudes,  attitude  toward  scientists,  attitudes  toward  a method  of 
teaching  science,  scientific  interests,  attitudes  toward  parts  of  the  curriculum  and  attitudes  toward  the  subject  of 
science. 

In  a recent  study,  Liu,  Hsieh,  Cho  and  Schallert  (2005)  investigated  the  effect  of  a problem-based  learning 
environment  on  middle  school  students’  self-efficacy,  attitudes  and  learning.  The  results  of  their  study  indicated  a 
positive  correlation  between  science  attitude  and  self-efficacy.  They  suggested  that  science  attitude  is  positively 
related  to  science  achievement. 
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Perceived  Usefulness  of  Science 

Shell,  Murphy,  and  Buring  (1989)  defined  the  perceived  usefulness  of  science  as  “how  does  one  perceive 
the  importance  of  science  for  achieving  various  life  goals”.  That  is,  ’’the  degree  to  which  a person  believes  that  using 
a particular  system  would  enhance  his  or  her  job  performance"  (Davis,  1989). 

Literature  showed  that  several  factors  can  affect  students’  perceived  usefulness  of  science.  According  to 
Chipman  and  Wilson  (1985),  career  expectations  and  career  goals  affect  the  perceived  usefulness  of  mathematics,  as 
may  the  adequacy  of  their  information  about  the  mathematical  requirements  of  career.  As  expected,  students' 
perceived  usefulness  of  mathematics  is  also  related  to  the  confidence  they  express  in  their  ability  (Lent,  Lopez,  & 
Bieschke,  1991).  Overall  these  studies  showed  that  self  efficacy  and  perceived  usefulness  are  related  to 
achievement. 

Science  Career  Interest 

Science  career  interest  is  enthusiasm  or  attention  for  a particular  career  in  science  (Super,  1984).  It  is  a 
learned  characteristic  that  can  be  developed  through  students’  participation  in  interactive  learning  events  with 
information  and  people  resources.  The  result  of  Koszalka’s  (1999)  research  on  science  career  interest  in  middle 
school  students  supports  Super’ s argument.  She  found  that  the  levels  of  science  career  interest  in  middle  school 
students  were  positively  related  to  the  richness  of  information  through  human  and  web  resources. 

To  promote  students’  science  career  interest,  research  indicated  that  efforts  should  be  made  on  different 
aspects.  When  students  lack  interests  in  science,  additional  mandatory  science  courses  will  not  likely  enhance  the 
development  of  their  interests.  Creating  a classroom  environment  that  nurtures  students’  interests,  curiosity  and 
confidence  in  their  own  abilities  might  be  more  effective  in  encouraging  the  increase  of  career  interest  in  science 
(Mason  & Kahle,  1988).  In  Lent,  et  al’s  (1991)  study,  they  suggested  self-efficacy  was  significantly  related  to  the 
interests.  They  argued  that  there  is  a reciprocal  relationship  between  self-efficacy  and  interest.  Accordingly,  science 
career  interest  can  be  promoted  by  enhancing  self-efficacy.  In  addition,  certain  instructional  and  behavioral 
intervention  strategies  seem  conducive  to  helping  student  pursue  science  courses  and  science-related  careers  (Mason 
& Kahle,  1988). 

Diagram  - The  Interrelationship  between  Student  Variables 


Metacognition 


Problem 
Solving  Skills 


Teacher  Variables 

Teachers’  teaching  practices  inarguably  have  strong  influence  on  student  learning.  By  performing  effective 
teaching  strategies,  the  teacher  can  contribute  a lot  to  the  development  of  students’  problem-solving  skills  and 
abilities.  Problem-based  learning  uses  constructivist  paradigm  to  support  its  framework,  so  in  this  part  of  the  article, 
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we  focus  on  some  teaching  practices  that  are  significant  to  constructivist  teaching  strategies.  The  teaching  practices 
of  interest  are  use  of  computers  and  the  Internet,  use  of  PBL  and  use  of  general  teaching  strategies.  From  the 
literature,  we  identify  the  following  relevant  teacher  characteristics  that  can  influence  these  teaching  practices: 
teachers’  computer  and  Internet  skills,  attitude  toward  the  use  of  web  resources,  feeling  of  preparedness  toward  the 
use  of  computer  and  the  Internet,  pedagogical  beliefs  and  science  teaching  efficacy.  Since  one  of  our  research  goals 
is  to  see  how  the  rate  of  KaAMS  use  will  influence  these  variables,  we  expect  to  develop  a model  that  suggests  how 
to  encourage  teachers  employ  more  of  problem-based  learning  approach  in  their  classroom. 

Use  of  Computers  and  the  Internet 

Technology  integration  has  important  implications  in  effective  teaching  and  learning.  It  is  expressed 
through  teachers’  use  of  computers  and  the  Internet,  defined  as  the  ways  that  teachers  use  computer/the  Internet  for 
teaching  their  classes  or  other  professional  activities.  In  this  section,  we  discuss  how  teachers’  use  of  computers  and 
the  Internet  is  influenced  by  different  teacher  characteristics  including  teachers’  computer  and  Internet  skills,  their 
attitude  toward  the  use  of  web  resources,  their  feeling  of  preparedness  to  use  computers  and  the  Internet  and  their 
pedagogical  beliefs. 

Computer  and  Internet  skills  refer  to  teachers’  knowledge  and  skills  of  using  computer  and  the  Internet  for 
their  teaching  (Becker,  2001).  Findings  from  the  Teaching,  Learning,  and  Computing  (TLC)  survey  in  1998,  a 
national  survey  of  more  than  4000  teachers  from  graders  4-12,  indicate  that  “the  ways  that  teachers  have  their 
students  use  computers  are  certainly  affected  by  their  own  level  of  technical  expertise”  (Becker,  2001,  p.  6).  More 
specifically,  “relevant  prior  computer  knowledge  may  be  an  important  pre-requisite  for  a teacher  to  make  the 
Internet  a valued  resource  in  their  classroom,  and  valuable  in  their  lesson  preparation  activities  in  particular” 

(Becker,  1999,  p.  29). 

Not  only  teachers’  computer  and  Internet  skills  influence  their  use  of  computers  and  the  Internet,  teachers’ 
attitude  as  well  as  their  feeling  of  preparedness  toward  using  computers  and  the  Internet  also  affect  how  teachers  use 
these  recourses.  Attitudes  toward  the  value  of  technology  use  in  class  have  been  shown  to  affect  teachers’  levels  of 
technology  implementation  (Bigatel,  2004).  Studies  indicate  that  there  are  a significant  positive  correlation  between 
attitudes  about  the  advantage  of  teaching  with  technology  and  levels  of  technology  implementation  (Becker,  1994; 
Lebruto,  2001). 

However,  results  from  the  Teaching,  Learning  and  Computing  (TLC)  survey  give  us  an  idea  that  that 
teachers’  beliefs  and  attitudes  about  technology  might  not  be  always  consistent  with  how  teachers  actually  feel  about 
using  technology.  While  nearly  90  percent  teachers  answered  they  would  consider  Internet  as  a valuable  resource  in 
the  TLC  survey,  not  that  many  teachers  reported  feeling  prepared  to  use  technology  as  statistics  from  National 
Center  for  Education  Statistics  (1999)  has  shown.  The  statistics  found  that  “less  than  a third  felt  very  well  prepared 
to  integrate  educational  technology  or  to  address  the  needs  of  students  with  limited  English  proficiency  or  from 
culturally  diverse  backgrounds,  or  with  disabilities”.  Teachers’  use  of  technology  is  significantly  related  to  their 
feelings  of  preparedness.  For  many  instructional  activities,  teachers  who  reported  feeling  better  prepared  to  use 
technology  were  a lot  more  likely  to  use  it  than  teachers  who  indicated  that  they  felt  unprepared  (Parsad,  Lewis,  & 
Farris,  2000).  Addressing  teachers’  feeling  of  preparedness  to  use  computer  and  the  Internet  is  therefore  very 
important  in  understanding  their  use  of  technology. 

What  teachers  believe  to  be  good  instructional  practices  also  affect  the  ways  teachers  use  technology  in 
their  classroom.  Becker  (1999)  found  that  teachers’  pedagogical  beliefs  are  one  of  the  major  predictor  of  teachers’ 
Internet  use  and  valuation. 

Use  of  PBL 

Under  use  of  problem-based  learning  we  understand  the  extent  to  which  teachers  use  teaching  strategies 
and  techniques  related  to  problem-based  learning  (e.g.  use  of  problem  as  a learning  task,  collaboration,  critical 
thinking,  etc.). 

Use  of  PBL  is  influenced  by  teachers’  pedagogical  beliefs.  Peterson  (2004)  mentioned  that  problem-based 
learning  could  only  be  effective  in  the  classroom,  when  the  teacher  departs  from  the  traditional  pedagogical 
paradigm  of  “teach,  learn,  practice,  and  assess”,  where  “the  teacher's  responsibilities  are  to  teach  and  to  assess  the 
learning.  The  students'  responsibilities  are  to  learn  and  to  demonstrate  that  they  have  learned  the  required 
information”  (p.  633).  Instead  the  teacher  becomes  a problem- solver,  who  is  actively  engaged  in  the  exploration  and 
discovery  of  the  solution  together  with  the  students.  A problem-based  classroom  removes  the  focus  from  the  teacher 
as  the  authority  figure  who  knows  all  the  answers,  and  puts  it  on  students  who  become  equal  partners  in  the  inquiry 
process. 
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Use  of  General  Teaching  Strategies 

Use  of  general  teaching  strategy  can  be  defined  as  how  often  teachers  use  a variety  of  teaching  strategies 
and  techniques  in  their  classrooms  (e.g.  questioning,  examples,  small  groups,  concept  maps).  According  to 
Grabowski,  Koszalka,  and  McCarthy  (1998),  general  teaching  methods  can  be  divided  into  the  following  six 
categories:  Present,  Guide,  Active  Learning,  Collaborative,  Problem-based  Learning  and  Role-Play.  We  can  argue 
that  Present  and  Guide  in  this  framework  belong  to  the  traditional  approach  and  Active  Learning,  Collaborative, 
Problem-based  Learning  and  Role-Play  methods  belong  to  the  constructivist  approach  where  students  learn  by 
doing. 

How  often  teachers  use  different  teaching  strategies  can  have  strong  implications  on  teachers’  use  of  and 
their  attitudes  towards  web-enhanced  PBL  in  science  education  in  middle  schools.  Kirby,  McGee,  Norris,  and 
Blaney,  (2003)  stated  that  technical  skills,  attitude  toward  technology,  constructivist  teaching  strategies,  and 
constructivist  teaching  philosophy  all  influence  constructivist  uses  of  technology.  By  constructivist  teaching 
strategies  and  philosophy  we  understand  teaching  that  organizes  student  work  around  meaningful  activities  so  that 
students  are  thoughtfully  engaged  with  content  (Ravitz,  Becker,  & Wong,  2000). 

Teachers'  beliefs:  Teachers’  beliefs  about  what  constitutes  good  instructional  practice.  Presumably,  their 
own  instructional  practices  reflect,  to  a large  extent,  what  they  believe  to  be  good  teaching,  their  beliefs  about  good 
teaching  reflect  their  understanding  about  how  students  learn  (Ravits,  Becker  & Wong,  2000).  Becker  (1999)  in  his 
study  on  relationships  between  teachers’  beliefs  and  teaching  practices  found  that  most  teachers  report  having  a 
balanced  combination  of  traditional  approaches  and  constructivist  approaches  in  their  classroom. 

Teacher  beliefs,  practices  and  computer  use  were  also  an  object  of  investigation  for  Riel  and  Becker 
(2000).  In  their  study  the  researchers  focused  on  the  differences  in  teacher  beliefs  and  practices  among  teacher 
leaders  and  average  teachers.  It  was  found  that  teacher  leaders  were  more  likely  to  use  constructivist  pedagogy  and 
computers  in  their  teaching.  Another  finding  relevant  to  the  current  study  was  that  science  teachers  had  one  of  the 
smallest  percentages  across  the  board  for  reporting  use  of  constructivist  practices  and  computers  in  their  classroom. 

Science  teaching  efficacy  is  an  essential  factor  that  impact  on  teacher’s  belief  system  and  their  sense  of 
confidence  as  it  relates  to  their  ability  to  be  successful  teachers  (Tschannen-Moran,  Hoy,  & Hoy,  1998).  It 
influences  personal  judgment  about  ability  and  competence  in  performing  tasks.  In  other  words,  it  influences  choices 
of  activities  as  well  as  the  amount  of  effort  expended  and  the  level  of  persistence  in  the  face  of  obstacles  (Wingfield 
& Ramsey,  1999).  It  has  been  correlated  with  various  measures  or  teacher  effectiveness,  including  classroom 
behaviors,  attitudes,  commitment  and  relations  to  classroom  (Evans  & Tribble,  1996). 

Diagram  - The  Interrelationship  between  Teacher  Variables 


Teacher  Characteristics  Teaching  Practice 


Significance  of  the  study 

This  study  aims  to  investigate  the  impact  of  web-enhanced  PBL  on  student  problem-solving  skills.  In 
addition,  a predictive  model  that  illustrates  the  interrelationships  between  various  variables  and  problem-solving 
skills  will  be  developed.  The  results  of  this  study  will  identify  factors  that  predict  students’  successful  problem- 
solving skills.  Knowledge  of  these  factors  will  enable  teachers  to  more  effectively  construct  learning  opportunities 
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for  students.  From  the  stakeholders’  standpoint,  identifying  factors  that  predict  teaching  practice  should  help  design 
teaching  workshops  for  in-service  teachers  as  well  as  programs  for  pre- service  teachers.  This  will  lead  to 
improvement  of  teacher  teaching  practice  and  ultimately  to  improvement  of  student  problem-solving  skills. 
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Abstract 

The  problems  we  encounter  in  everyday  life  are  complex  and  ill- structured.  However,  the  problems  in  a 
class  are  well- structured  and  easily  solved  as  long  as  students  follow  the  instructions.  Considering  that  one  of  the 
important  goals  of  education  is  to  help  students  become  successful  problem  solvers,  we,  as  instructional  designers 
need  to  develop  new  instructional  models  and  methods  to  support  the  students  ’ problem  solving.  Here  we  suggest  a 
design  model  to  help  students  become  successful  diagnosis-solution  problem  solvers. 

Introduction 

We  encounter  a variety  of  problems  in  everyday  life.  These  problems  are  usually  ill-structured,  complex 
and  dynamic  as  well  as  domain  and  context  specific  (Jonassen,  2004).  However,  in  many  cases,  our  educators  do  not 
consider  the  characteristics  of  the  real  problems  in  our  classes.  They  assume  that  problem  solving  skills  can  be 
transferred  from  teachers  or  technologies  to  students  (Jonassen,  1999).  Another  common  belief  is  that  students  can 
apply  simple  problem  solving  skills  to  complex  problem  contexts.  Consequently,  they  have  focused  on  knowledge 
delivery  or  sequencing  of  content  (Jonassen,  1999)  in  instructional  design. 

However,  problem  solving  skills  are  constructed  by  learners  themselves  based  on  the  individual  learner’s 
interpretations  and  practices.  Developing  the  learner’s  own  mental  models  about  the  problems  is  essential  to 
promote  effective  problem  solving.  In  order  to  be  successful  problem  solvers,  learners  should  be  able  to  manipulate 
and  test  their  mental  models  for  the  problems  (Jonassen,  2004). 

Also,  well- structured  problem  solving  skills  are  difficult  to  transfer  to  complex  and  ill- structured  problems. 
This  is  because  the  necessary  skills  to  solve  problems  between  ill- structured  and  well- structured  problems  are 
different  from  each  other.  Shin,  Jonassen,  and  McGee  (2003)  found  that  students’  problem  solving  skills  for  ill- 
structured  problems  are  different  from  the  problem  solving  skills  for  well- structured  problems  in  the  science 
domain.  According  to  them,  the  domain  knowledge  and  justification  skills  are  significantly  related  to  solving  well- 
structured  problems  and  the  domain  knowledge,  justification  skills,  science  attitudes,  and  regulation  of  cognition 
skills  are  significantly  related  to  solving  ill- structured  problems. 

We  argue  the  constructivist  learning  environment  (CLE)  supports  students  to  become  successful  problem 
solvers.  It  provides  real  or  authentic  problems  and  requires  students  to  engage  in  problem  solving  processes.  Also, 
the  CLE  provides  related  cases  with  the  problems  so  that  students  can  approach  the  problems  in  a variety  of  ways. 

By  doing  so,  students  understand  different  ways  to  approach  the  same  problems  and  expand  their  mental  models. 

The  purpose  of  this  paper  is  to  present  how  a constructivist  learning  environment  can  be  designed.  We  chose  a 
diagnosis-solution  problem  to  design  a CLE.  In  this  paper,  we  will  present  both  specific  processes  in  designing  the 
CLE  and  the  visual  outcomes  of  the  specific  processes. 

Learning  Context  and  Instructor’s  Needs 

Learning  Context 

This  course,  “Occupational  Therapist:  Developmental  Framework”,  is  offered  by  the  Department  of 
Occupational  Therapy  in  a mid-west  university.  Most  of  the  students  in  the  class  are  juniors.  The  class  is  based  on  a 
block  system  that  is  similar  to  medical  school.  Each  block  consists  of  8-10  students  and  is  a 7- week  intensive  course. 
Lecture  and  laboratory  coursework  is  designed  to  provide  the  occupational  therapy  (OT)  student  with  an 
understanding  of  the  process  of  normal  and  delayed  development  and  to  prepare  the  students  to  administer,  score, 
and  interpret  results  from  common  assessments  intended  to  evaluate  the  development  of  infants  and  young  children. 

Instructor’s  Needs 

Many  students  don’t  know  how  to  participate  in  the  case  study.  Also,  they  don’t  know  how  to  systemically 
approach  the  case  and  how  to  make  rationale  explanations  about  their  decisions.  In  addition,  students  don’t 
understand  what  experts  they  need  to  contact  to  solve  the  problem.  However,  the  instructor  wants  students  to  know 
what  things  really  happen  in  the  field  when  students  go  out  in  the  field  as  occupational  therapists. 
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Considering  the  gap  between  students’  current  status  and  the  instructor’s  expectation,  our  learning  environments 
aimed  to  help  students  to  become  an  effective  problem  solver  in  occupational  therapy.  More  specifically  we 
supported  students  to  be  a successful  problem  solver  by  providing  them  opportunities  to  manipulate  their  mental 
models  for  the  problem,  which  is  effective  to  develop  mental  models  (Jonassen,  2004).  In  our  learning  environment 
students  will  achieve  the  following  specific  goals; 

1.  Students  will  be  able  to  distinguish  the  patients’  developmental  processes  (normal  & abnormal). 

2.  Students  will  be  able  to  understand  the  experts  that  occupational  therapists  interact  with  in  the  fields. 

3.  Students  will  be  able  to  systemically  approach  the  cases. 

4.  Students  will  be  able  to  judge  what  would  be  an  appropriate  diagnostic  test 

Theory  Base 

The  main  theories  used  for  this  learning  environment  were  activity  theory,  cognitive  flexibility  and 
constructivistism.  We  will  describe  how  these  theories  have  applied  in  our  CLE  based  on  the  client’s  needs  and 
characteristics  of  the  course. 

The  main  goal  of  this  course  Occupational  Therapists  (OT)  4410  was  that  students  will  be  able  to  have  their 
own  mental  models  about  the  role  of  occupational  therapists  in  the  real  field.  Students  were  expected  to  build  their 
own  mental  models  about  how  they  approach  the  cases  and  how  they  interact  with  other  experts  in  the  field. 

We  decided  to  make  boundaries  for  this  learning  environment  because  we  could  not  cover  everything  in 
our  learning  environment.  We  needed  to  know  what  the  important  elements  were  and  how  occupational  therapists 
function  in  the  field.  In  order  to  make  boundaries  for  this  learning  environment  that  satisfied  the  learning  goal,  we 
conducted  task  analysis  procedures  using  Activity  theory.  We  chose  Activity  theory  as  a framework  to  analyze  how 
first  year  occupational  therapists  function  in  the  field  for  two  reasons.  First,  Activity  theory  provides  a framework 
for  analyzing  both  individual  and  social  levels  interlinked  at  the  same  time  (Mwanza,  2001).  Since  the  occupational 
therapists  work  individually  as  well  as  with  others  (e.g.,  speech  therapist,  physicians,  social  workers),  the  activity 
theory  was  appropriate  to  analyze  how  individual  occupational  therapists  function  in  the  field.  Second,  Activity 
theory  focused  on  “interaction  of  human  activity  and  consciousness  (the  human  mind  as  whole)  within  its  relevant 
environmental  context”  (Jonassen,  Tessmer,  & Hannum,  1999).  The  Activity  theory  task  analysis  provided  us 
specific  information  about  how  an  individual  functions  in  the  field.  Since  students  do  not  exactly  know  about  their 
roles  in  the  field,  the  analysis  results  will  be  helpful  for  students  to  understand  how  occupational  therapists  interact 
with  other  elements  in  the  field.  The  most  important  things  we  found  in  the  task  analysis  were  that  in  order  to  be  a 
successful  problem  solver  in  occupational  therapy,  students  should  develop  their  own  mental  models  about 
developmental  domains  and  body  systems  of  the  patients  as  well  as  develop  mental  models  about  other  experts’ 
roles  and  how  other  experts  interact  with  the  occupational  therapists. 


In  order  to  support  students’  mental  model  development  about 
problems,  we  decided  to  use  simulation.  Simulation  is  very 
effective  to  have  students  engage  in  the  problem  solving  (Savery 
& Duffy,  1994)  and  help  students  develop  their  own  mental 
models  by  practicing  authentic  problems.  The  simulation 
activities  were  created  based  on  an  experts’  mental  model  about 
ill-structured  problem  solving  in  occupational  therapy.  Figure  2 
represents  one  of  the  experts’  mental  models  about  problem 
solving.  In  our  learning  environments,  students  are  given  cases 
with  pictures  and  video  clips,  which  make  the  learning  more 
authentic  and  engaging  (Merseth,  2000). 
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Once  students  are  given  problems,  they  are  asked  to  analyze 
developmental  issues,  identify  body  systems  relevant  to 
developmental  domains  and  clinically  judge  the  patients’ 
concerns  and  strengths.  Then,  students  are  asked  to  choose 
an  appropriate  diagnosis  test  tool  for  further  treatment.  In 
our  simulation  environments,  we  focused  on  two  things: 
problems  and  different  experts’  perspectives.  First,  we 
focused  on  development  of  cases.  Our  cases  were  generated 
by  the  expert  and  they  all  satisfy  Jonassen  (1999)’s 
recommendations  of  problems  for  authentic  learning:  1) 
Problems  should  be  authentic  and  challenging  so  that 
learners  will  have  an  ownership  of  the  problems.  2) 
Problems  should  describe  performance  environments  and  all 
relevant  stakeholders  which  make  the  learning  authentic.  3) 
Problems  should  support  student  manipulation  of  the 
problems. 


Figure  3 


Second,  we  focused  on  presenting  different 
perspectives  of  experts  relevant  to  the  occupational 
therapists:  other  occupational  therapists,  speech 
therapists,  physical  therapists,  physicians  and  social 
workers.  We  argued  presenting  different 
perspectives  is  helpful  for  students  to  develop  their 
own  mental  models  about  how  occupational 
therapists  function  in  the  community. 


Multiple  perspectives  will  make  students  review  their  existing  perception  of  occupational  therapists  and 
view  the  problems  from  the  landscape  criss-crossing  perspectives  (Spiro,  Coulson,  Feltovich,  & Anderson,  1988; 
Spiro,  Feltovich,  Jacobson  & Coulson,  1992).  We  argue  that  this  process  will  promote  students’  conceptual  change 
about  the  roles  of  occupational  therapist  as  well  as  provide  understanding  about  their  roles  in  the  field. 

Ultimately,  this  will  help  students  develop  their  own  mental  models,  which  are  authentic.  Also,  multiple 
perspectives  make  it  possible  for  students  to  revisit  the  same  material  over  and  over  again.  By  doing  so,  the 
possibility  of  missing  important  concepts  will  be  reduced  and  provide  new  insights  about  the  problems  (Spiro, 
Coulson,  Feltovich,  & Anderson,  1988;  Spiro,  Feltovich,  Jacobson  & Coulson,  1992).  This  will  contribute  to 
developing  mental  models  which  are  effective  for  solving  ill- structured  problems  in  occupational  therapy. 


Design  Process 

The  design  processes  were  described  as  follows.  Although  our  design  processes  were  described  as  linear, 
the  actual  processes  were  very  dynamic  and  were  similar  to  prototype  design  processes.  Each  design  process  will  be 
described  in  detail. 


Analysis  of  client’s  Needs 

We  regularly  met  with  the  client  every  Tuesday  at  9:45  in  the  occupational  therapy  conference  room.  The 
meetings  with  the  client  provided  us  with  understanding  about  what  the  client  wanted  us  to  develop  for  her  course. 
In  the  meeting  we  communicated  with  the  instructor  to  find  out  what  she  expected  from  the  course,  what  objectives 
she  had  in  mind,  what  limitations  we  had,  what  resources  the  department  had  and  how  much  they  could  support  us. 
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Figure  4 


Deciding  design  Concepts 

Based  on  the  analysis  of  client’s  needs,  we  were  able  to  articluate  design  concepts  for  the  learning 
environment.  She  wanted  the  students  to  be  a good  problem  solver  as  new  occupational  therapists  in  the  field.  More 
specifically,  she  wanted  students  to  have  understanding  about  other  experts  in  the  field  and  to  learn  how  the 
occupational  therapists  interact  with  them.  In  order  to  support  this  client’s  needs,  we  decided  to  provide  students 
with  opportunities  to  simulate  the  occupational  therapist’s  roles  with  cases.  Also,  we  supported  students  with 
different  levels  of  case  difficulty  and  with  different  relevant  experts’  perspectives  . 


Designing  Constructivist  Learning  Environments 


Figure  5 
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Task  Analysis 

Task  analysis  was  essential  to  developing  the  constructivist  learning  environment.  We  conducted  Task 
Analysis  with  the  use  of  Activity  Theory  analysis.  This  is  because  Activity  theory  provided  a theoretical  framework 
for  analyzing  both  individual  and  social  levels  interlinked  at  the  same  time  (Mwanza,  2001).  Based  on  the  Activity 
Theory  analysis,  we  drew  boundaries  for  our  CLE  and  considered  what  components  should  be  integrated  in  our 
CLE:  We  assumed  the  subject  in  the  system  would  be  the  first  year  occupational  therapist,  they  would  interact  with 
other  experts  based  on  understanding  of  their  division  of  labors  and  they  had  to  understand  what  kind  of  insurance 
and  social  security  supported  their  patients. 

Learners’  Analysis 

To  begin  our  analysis  of  learners,  we  observed  and  videotaped  a class  session  focused  on  a case  study. 
Based  on  the  classroom  observation,  we  were  able  to  identify  how  the  case  study  was  implemented,  what  roles 
students  played  and  the  role  the  instructor  played  in  the  case  study.  The  observation  showed  students  were  not 
familiar  with  case  studies,  and  they  needed  more  structured  case  studies.  The  rubric  the  instructor  provided  to 
students  was  not  clear,  and  there  were  not  enough  systematic  supports  for  students  to  develop  arguments  about  their 
decisions. 

At  the  same  time,  in  order  to  identify  students’  current  mental  models  about  the  problems,  we  conducted 
learner  analysis  by  interviewing  students.  This  process  was  necessary  to  understand  the  students’  current  mental 
models  for  this  course,  which  then  informed  the  researchers  about  the  degree  of  scaffolding  needed  in  the  CLE.  We 
interviewed  two  students  with  the  help  of  the  instructor:  one  student  was  struggling  with  the  class  format  and  the 
other  student  was  not.  We  found  both  students  commonly  had  less  knowledge  about  diagnostic  tests.  Also,  the 
struggling  student  displayed  difficulties  connecting  the  developmental  domains  of  the  patient  and  to  the  patient’ s 
body  systems.  This  showed  that  we  needed  to  focus  on  supporting  students’  understaning  of  how  body  systems  are 
relevant  to  the  developmental  process  and  what  diagnostic  tests  are  appropriate  for  patients. 

Developing  2nd  Prototype 

We  improved  our  prototype  based  on  the  discussion  with  the  group  members  and  the  client.  We  presented 
the  case  with  via  video  clips  to  improve  students’  engagement  in  the  problem  and  also  decided  to  provide  different 
perspectives  with  videos  rather  than  text  based  narration.  This  decision  came  from  the  fact  that  providing  cases  with 
an  interactive  technology  were  particularly  exciting  (Merseth,  2000). 

Discussion  and  Evaluation 

We  met  with  the  client  once  a week.  In  addition  to  regular  meetings,  we  also  sent  emails  to  the  instructor, 
asked  her  questions  and  requested  her  to  provide  scenarios  or  other  cognitive  resources  (e.g.,  instrument  pictures). 
After  the  regular  meetings,  our  team  members  had  follow-up  meetings.  The  discussions  with  the  group  members  led 
to  revisions  of  the  1st  and  2nd  prototypes.  We  focused  on  multiple  navigations,  considered  how  to  scaffold  students’ 
learning  and  considered  how  students’  performances  would  be  evaluated.  Lastly,  we  developed  rubrics  for  students’ 
evaluations.  Since  the  instructor  wanted  to  help  students  develop  a systems  approach,  we  developed  the  rubric  based 
on  the  interview  with  her. 

Finalizing  the  prototype 

We  finalized  the  current  prototype.  The  finalized  version  of  prototype  provided  three  cases,  different 
perspectives  with  video  clips,  scaffolding  system,  supporting  evaluation  of  relevant  body  system  with  developmental 
domains  and  clinical  judgment  and  choosing  an  appropriate  diagnostic  test.  In  particular,  we  focused  on  scaffolding 
students’  development  of  multiple  perspectives  by  prompting  and  asking  them  to  make  justify  their  decisions.  In 
addition,  same  ways’  scaffolding  was  provided  in  the  process  of  clinically  evaluating  the  body  parts  of  the  patients 
and  choosing  an  appropriate  diagnostic  test. 


Product  Description 

Considering  the  client’s  needs  and  the  results  of  learner  analyses,  it  was  decided  to  integrate  the  following 
three  components  in  the  Constructivist  Learning  Environment:  simulation,  multiple  perspectives  and  argument 
(scaffolding). 
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The  intro  page  (Figure  6)  started  with  instructor’s  welcome  statement  and  the  explanation  of  the  class. 
Students  will  know  the  objectives  of  the  class  and  what  cases  they  will  use  in  the  class.  The  case  study  (Figure  7) 
starts  from  a normal  occupational  therapist’s  desk.  On  the  desk,  students  will  see  case  binder,  telephone  messages 
from  other  relevant  experts  (e.g.,  speech  therapist,  physician,  social  worker  and  so  on)  and  web  resources. 


Simulation 

A simulation  was  developed  which  describes  how  first-year  occupational  therapists  interact  with  other 
community  members  and  how  they  diagnose  a patient’s  status  based  on  five  developmental  areas:  motor,  adaptive 
behaviors,  social/emotional,  language  and  cognitive  development.  Also,  we  simulated  how  first-year  occupational 
therapists  choose  the  most  appropriate  test  for  further  medical  treatments.  Students  are  supposed  to  approach  the 
case  using  the  following  six  procedures; 

1 . Analyze  the  family  background 

2.  Analyze  other  experts’  perspectives 

3.  Analyze  patients’  overall  developmental  processes 

4.  Determine  what  parts  of  the  body  systems  are  relevant  to  the  developmental  processes 

5.  Clinically  judge  and  evaluate  patients’  strengths  and  concerns  in  terms  of  body  parts 

6.  Choose  an  appropriate  diagnostic  test 


Argumentation  Scaffolding 

Argumentation  scaffolding  was  constructed  to  help  students  develop  their  own  mental  models  about  the 
problems,  which  are  cases  in  this  learning  environment.  Students  are  supposed  to  participate  in  all  the  processes  in 
the  simulation.  In  each  step,  prompt  questions  were  provided  in  order  to  facilitate  systemic  thought  processes  in  the 
students.  A structured  system  approach  was  provided,  which  was  the  above  six  processes.  In  each  step,  students  will 
be  asked  to  make  decision  and  provide  their  own  rational  for  their  decisions.  For  example,  they  will  be  asked,  “what 
experts  are  you  going  to  contact  to  approach  the  problem?”  Once  students  choose  the  experts,  they  will  be  asked  to 
explain  why  students  chose  those  experts  (See  assessment  to  view  visual  picture  of  argumentation  scaffolding) 


Multiple  Perspectives:  In  real  situations, 
occupational  therapists  interact  with  other  experts 
(physical  therapists,  psychologists,  speech  or 
language  therapist,  doctors,  institutions,  families 
and  so  on)  when  they  approach  the  problems. 
Therefore,  they  need  to  understand  how  each 
expert  approaches  the  same  problems  from  the 
different  perspectives.  In  order  to  support  this 
reality  in  the  field,  other  experts’  perspectives  are 
provided  to  students  with  the  use  of  video  clips. 


Figure  8 
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Assessment 

Our  evaluation  method  was  a writing  assignment.  The  purpose  of  our  assignment  was  to  have  students 
experience  expert’s  thinking  process.  Since  there  is  mental  gap  between  experts  and  novices,  we  scaffolded  students 
by  using  prompts. 
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Questions  for  Treatment  Plan 


1. )  Based  on  your  interviews  and  background  information,  which  of  the  developmental  domains  should 
be  focused  on  for  Marcelo's  treatment  plan?  From  the  list  below,  select  ALL  that  apply: 

□ Motor 

O Adaptive  Behavior 

□ Social/Emotional 

□ Language 

□ Cognitive 

2. )  What  areas  will  you  evaluate  in  Marecelo's  case?  From  the  list  below,  select  all  that  apply  and  justify 
each  of  your  selections  in  the  text  areas  below  (be  sure  to  relate  your  justifications  to  the  developmental 
domains  that  you  selected  in  QUESTION  ONE): 


□ Muscle  strength  (if  checked,  justify  your  response  below) 


I I Postural  alignment  (if  checked,  justify  your  response  below) 


□ Movement  patterns  (if  checked,  justify  your  response  below) 


Figure  10 


For  example,  students  watched  the  videos  in  order  to 
learn  how  other  experts  approach  the  problem.  Essay 
questions  and  multiple  choice  questions  were  given  to 
help  students  perceive  what  things  they  need  to 
understand  in  the  video.  The  essay  questions  asked 
students  to  make  jsutifications  about  their  arguments. 
At  the  same  time,  multiple  questions  also  asked 
students  to  provide  their  arguments  about  their 
choices. 


In  addition,  in  the  process  of  clinical  judgment  and 
identifying  body  system  relevant  to  development 
domains,  scaffolding  was  given.  Students  were  also 
required  to  make  decisions  by  choosing  the  answers 
and  making  justifications  about  their  decisions.  Since 
the  course  curriculum  follows  the  medical  block 
system,  which  is  a short  quarter,  and  the  number  of 
students  in  the  class  is  small,  we  felt  writing  evaluation 
was  appropriate. 


Discussion 

We  will  discuss  how  our  design  model  is  different  from  traditional  design  models.  Compared  to  the 
traditional  design  model  (e.g.,  ADDIE),  our  model  is  very  flexible  and  is  a prototype  centered  design.  Rather  than 
waiting  for  all  the  analyses  to  be  finished,  we  first  developed  the  prototype  based  on  small  understanding  and  basic 
design  concept.  Task  analyses,  learner  analyses,  meetings  with  the  client,  discussions  with  the  group  members  and 
evaluations  of  the  prototype  were  conducted  simultaneously  to  improve  the  1st  prototype.  Regular  meetings  with  the 
client  and  discussions  with  the  client  and  group  members  were  very  helpful  to  improve  the  initial  prototype. 
Although  there  had  been  several  changes  in  our  design,  we  kept  our  design  principles  we  planned  at  first.  In 
addition,  it  was  helpful  to  develop  a simple  prototype  and  add  new  designs  based  on  the  discussion.  Therefore,  we 
were  able  to  save  time  and  energy.  The  client’s  needs  could  be  easily  applied  in  the  prototype,  which  would  be  hard 
in  the  traditional  ADDIE  model. 


Limitations 

There  are  several  limitations  of  our  design  research.  First,  we  were  not  able  to  conduct  a formative 
evaluation  and  had  no  opportunity  to  revise  the  online  CLE.  If  we  were  able  to  conduct  the  formative  evaluation,  we 
would  likely  identify  other  things  we  need  to  consider  in  this  environment  and  how  the  online  CLE  can  be 
improved.  Second,  it  was  unfortunate  for  us  that  this  course  was  opened  only  in  the  winter  semester.  In  order  to 
conduct  the  user  evaluation  for  summative  evaluation,  we  needed  to  wait  until  the  instructor  opens  the  class  again. 
Third,  although  it  was  out  of  our  responsibility,  it  would  be  more  reasonable  for  us  to  further  develop  the  CLE  to 
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require  students  to  provide  appropriate  treatments  for  the  patients.  However,  the  curriculum  of  the  course  only 
covered  the  intial  examination  of  the  patient  selection  of  an  appropriate  diagnostic  test.  We  were  not  able  to  develop 
the  next  process,  which  is  important  in  developing  diagnosis  problem  solving  skills. 
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Abstract 

As  online  learning  become  prevalent,  supporting  students'  self -regulation  is  significantly  more 
important.  Online  learning  is  different  from  face-to-face  learning  in  that  it  offers  a lot  more  learner  autonomy  but 
at  the  same  time  also  demands  more  responsibility  from  them.  Therefore,  instructional  designers  and  instructors 
should  encourage  students  to  self -regulate  their  learning.  This  paper’s  purposes  are  to  distinctly  define  self- 
regulated  learning  in  online  learning  contexts,  suggest  an  instructional  framework  for  instructional  designers  and 
online  instructors  to  use  to  scaffold  students’  self -regulation  in  online  learning  and  finally,  recommend  strategies 
to  support  self -regulation  in  online  learning. 


Introduction 

As  online  learning  becomes  prevalent,  supporting  students’  self-regulation  is  becoming  significantly 
important.  Online  learning  is  different  from  face-to-face  learning  in  that  it  offers  a lot  more  learner  autonomy  but  at 
the  same  time  also  demands  more  responsibility  from  them.  In  addition,  online  learning  is  usually  mediated  by  web- 
based  asynchronous  communication  tools  such  as  discussion  boards  that  can  not  provide  verbal  and  non-verbal 
communication  (Hron  & Friedrich,  2003).  Due  to  these  limitations  of  online  learning  environments,  instructors  are 
therefore  not  able  to  offer  students  immediate  and  direct  support  which  makes  promoting  self-regulated  learning 
even  more  difficult. 

As  a way  to  promote  students’  self-regulated  learning  in  online  environments,  various  instructional 
strategies  has  been  recommended  such  as  embedding  cognitive  or  meta-cognitive  questions  into  learning  materials 
(Shin,  1997),  displaying  self-reflection  support  tools,  e.g.,  online  journals,  online  time  and  task  management  tools 
(Cho,  2004),  and  giving  note-taking  assignments  (Lim,  2002).  However,  these  approaches  have  been  criticized  since 
they  do  not  consider  the  learning  context  in  which  the  learning  takes  place  (Meyer  & Turner,  2002).  Therefore, 
recent  research  (Butler,  2002;  Meyer  & Turner,  2002)  has  begun  to  acknowledge  the  learning  context  as  a critical 
factor  to  consider  when  promoting  self-regulation  in  learning.  Unfortunately,  few  have  been  able  to  suggest 
instructional  approaches  for  promoting  online  self-regulation  within  the  learning  context. 

According  to  some  researchers  (Meyer  & Turner,  2002;  Perry,  VandeKamp,  Mercer,  & Nordby,  2002; 
Zimmerman,  Bonner,  & Kovach,  1996),  an  instructional  approach  is  scaffolding.  Hence,  this  paper’s  purposes  are  to 
distinctly  define  self-regulated  learning  in  online  learning  contexts,  suggest  an  instructional  framework  for 
instructional  designers  and  online  instructors  to  scaffold  students’  self-regulation  in  online  learning  and  finally, 
recommend  strategies  to  support  self-regulation  in  online  learning. 


Self-regulated  and  co-regulated  learning 

Self-regulated  learners  are  defined  as  individuals  who  are  able  to  effectively  orchestrate  cognitive,  meta- 
cognitive  strategies,  resource  management  and  motivation  in  learning  (Zimmerman,  1990).  Self-regulated  learners 
have  knowledge  about  themselves  as  learners  and  knowledge  about  learning  strategies  (Heo,  2000).  They  know  how 
to  use  the  learning  strategies  to  achieve  their  goals  depending  on  the  learning  task.  They  are  also  persistent  and  make 
efforts  to  master  the  learning  with  self-efficacy.  Self-regulated  learners  take  responsibility  for  their  learning  (Shin, 
1997)  and  effectively  use  their  learning  resources,  e.g.,  time  management,  seek  help  and  structure  their  learning 
environments.  They  are  reflective  learners  through  self-imposed  questions,  such  as  what  did  I learn,  what  learning 
strategies  are  most  suitable  to  achieve  my  goals,  how  much  time  should  I spend  so  on  and  so  forth. 

More  recently,  researchers  (Meyer  & Turner,  2002;  McCaslin  & Good,  1996;  McCaslin  & Hickey,  2001; 
Perry,  VandeKamp,  Mercer,  & Nordby,  2002)  have  argued  that  context  should  be  considered  in  the  self-regulated 
learning.  Their  argument  is  that  learners  co-construct  knowledge  through  social  interaction.  As  a result,  through 
these  interactions  they  co-regulate  with  their  peers  and  instructors  to  achieve  shared  goals.  According  to  Butler 
(2002),  “it  is  impossible  to  judge  individuals’  capacity  to  self-regulate  without  consideration  of  context  (p.60)” 

Thus,  it  is  believed  the  class  or  group  should  co-regulate  as  a whole  through  negotiation  and  sharing. 
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Figure  1 displays  these  two  types  of  self-regulation:  individual  regulation  and  group  co-regulation. 
Furthermore,  we  can  then  apply  this  to  the  setting  for  online  learning  environments  where  the  context  (e.g., 
technology,  activities,  and  tasks)  mediate  learners’  self-regulation  and  learning  outcomes.  The  basic  assumptions  in 
online  learning  environments  are  that  knowledge  is  constructed  through  sharing  and  negotiation,  and  learning  occurs 
through  learners’  interactions. 


Figure  1 : Model  of  Self-regulation 


In  online  learning  environments,  individual  learners  not  only  need  to  engage  in  self-regulation  but  also, 
depending  on  the  task  and  activities’  nature,  they  are  sometimes  required  to  co-regulate  with  others.  In  other  words, 
learners  should  have  the  capability  to  self-regulate  as  well  as  co-regulate  with  others  when  necessary. 

Individual  learners  in  context 

Self-regulated  learners  in  this  situation  are  those  who  have  knowledge  about  themselves  as  well  as 
cognitive,  meta-cognitive,  resource  management,  and  motivational  strategies.  Once  they  encounter  problems  to 
solve,  they  know  what  they  need  to  do  and  apply  what  they  know  into  the  problem  solving  process.  They  will  reflect 
on  what  they  know  and  what  they  do  not  know.  Then,  they  will  set  goals  and  specific  cognitive  and  resource 
management  strategies  to  solve  problems.  They  regularly  monitor  and  evaluate  their  learning  process  and  change 
their  learning  strategies  and  goals  if  necessary.  Self-regulated  learners  in  this  situation  will  give  ceaseless  effort  to 
achieve  their  goals  through  self-efficacy. 

Group  of  learners  in  context 

Self-regulated  learners  in  this  situation  are  those  who  have  knowledge  about  themselves  and  self-regulated 
learning  strategies.  In  addition,  self-regulated  learners  in  this  situation  are  aware  they  need  to  collaborate  and  co- 
regulate  with  other  group  members.  They  proactively  participate  in  social  negotiations  and  knowledge  construction 
processes.  They  should  possess  the  social  skills  to  co-regulate  with  other  members,  be  able  to  communicate, 
encourage,  and  share  information  in  order  to  collaboratively  work  together.  Through  this  social  self-regulation 
process,  they  establish  relationships  with  other  members  and  contribute  to  the  development  of  the  self-regulation 
circumstances  in  the  online  learning  environments. 

Self-regulation  in  online  learning 

We  defined  self-regulated  learners  in  online  leaning  environments  as  those  who  have  knowledge  about 
themselves  as  learners  or  as  independent  learners  and  collaborators,  and  knowledge  about  learning  strategies  which 
are  necessary  for  individual  and  co-regulation.  They  proactively  participate  in  learning  activities,  namely, 
negotiating,  sharing,  and  collaboration  with  an  open  mind  and  self-efficacy.  Considering  the  definition  of  self- 
regulated  learning  and  the  characteristics  of  online  learning  environments,  we  suggest  a framework  explaining  how 
self-regulated  learning  occurs  in  online  learning  environments. 

As  shown  in  Figure  2,  in  online  learning  environments,  the  fundamental  components  are  the  individual 
learner  and  the  group  of  learners.  In  addition,  self-regulated  learning  is  mediated  by  the  context  and  the  learning 
processes,  such  as  individual  and/or  co-regulation.  The  expected  learning  outcomes  are  categorized  into  the 
individual  learning  outcomes  and  the  group  learning  outcomes.  We  will  explain  further,  in  detail,  each  element  of 
self-regulation  in  online  learning  environments. 

Learners 

Learners  are  the  basic  components  in  self-regulated  learning.  Self-regulated  learners  should  be  able  to  self- 
regulate  as  well  as  co-regulate  depending  on  the  nature  of  the  instructional  task  and  activities.  Self-regulated 
learners  as  individual  learners  have  sophisticated  epistemological  beliefs,  high  cognition  levels,  meta-cognition, 
resource  management  skills  and  a strong  motivation  to  learn.  Epistemological  beliefs  are  learners’  perceptions  about 
knowledge  and  learning.  Learners  who  have  sophisticated  epistemological  beliefs  are  thought  to  use  more  cognitive 
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and  meta-cognitive  strategies  (Hofer,  2004).  They  have  knowledge  about  learning  strategies  and  these  strategies  are 
chosen  through  their  meta-cognition.  In  order  to  solve  problems,  they  chose  appropriate  learning  strategies,  apply 
them  into  problem  solving  process  and  evaluate  whether  the  application  of  the  learning  strategies  are  effective.  Self- 
regulated  learners  are  able  to  self-monitor  how  they  manage  time,  know  who  to  ask  when  they  seek  help  and  are 
aware  of  appropriate  and  effective  strategies  in  learning.  Once  they  understand  the  tasks  and  set  their  goals,  they  will 
persistently  make  efforts  in  the  problem  solving  process  through  high  self-efficacy.  Self-regulated  learners  as  a 
group  of  learners  are  involved  in  the  co-regulation  process.  Co-regulation  occurs  through  co-cognitive,  co-meta- 
cognitive  and  co-resource  management  processes  in  order  to  reach  common  goals.  During  the  co-regulation 
processes,  the  group  of  learners  share  meaning  making,  negotiate,  build  consensus,  and  co-construct  knowledge. 
Through  this  meaning  and  knowledge  making  process,  they  establish  social  relationships.  They  know  who  they  are 
amongst  a group  of  learners  and  contribute  to  the  formation  of  social  presence  in  the  learning  environment. 
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Figure  2:  Framework  for  Self-regulated  Learning  in  an  Online  Environment 


Mediators 

Context  and  learning  process  are  mediators  of  the  self-regulated  learning  process.  Context  consists  of 
technology,  task,  feedback,  assessment,  and  interaction,  and  the  learning  process  consists  of  individual  regulation, 
co-regulation  as  well  as  both  individual  and  co-regulation.  More  specifically,  the  context  determines  the  learning 
process.  In  particular,  the  characteristics  of  the  tasks  and  assessment  strongly  influence  the  students’  learning 
process.  Web-based  tools,  such  as,  email,  listservs,  discussion  boards,  instant  messenger,  video  and  audio 
conferencing,  3D  environments,  and  delivery  systems  provide  the  medium  for  communication.  Also,  in  addition  to 
interaction,  feedback  which  is  both  internal  and  external  all  contributes  to  this  mediation  process.  Activities  come 
from  the  learning  tasks  and  assessment  and  they  influence  the  type  of  learning  process. 

Online  learning  outcomes 

Learning  outcomes  result  from  the  learning  process.  Individual  outcomes  are  the  results  of  individual 
regulation  and  group  outcomes  are  results  of  co-regulation.  Through  the  process  of  self-regulated  learning, 
individual  learners  will  acquire  competency  and  understanding  related  to  the  task,  be  able  to  perform  reflective 
thinking  and  also  will  become  strongly  motivated.  As  for  the  group  of  learners,  co-constructed  knowledge  will  result 
from  the  co-regulation  process  among  the  group  as  they  participate  in  shared  meaning-making,  negotiation  and 
consensus  building.  Last,  social  relationships  will  also  develop  from  their  interaction  and  collaboration. 

Scaffolding  students’  self-regulation 

The  basic  assumption  in  self  regulation  learning  is  that  students  have  knowledge  about  themselves  as 
learners  and  knowledge  about  self-regulated  learning  strategies.  However,  not  all  students  will  possess  knowledge 
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about  themselves  as  learners  or  knowledge  about  self-regulated  learning  strategies  (Heo,  2000).  Therefore, 
instructional  designers  should  focus  on  how  to  scaffold  students’  knowledge  about  themselves  as  learners  and 
knowledge  about  self-regulated  learning  strategies. 

Table  1.  Scaffolding  Strategies  to  Support  Self-regulated  Learning 

Categories  Source  Scaffolding  strategies 

Tasks  Internal  Provide  students  with  the  opportunity  to  set  their  goals  in  the  class 

Provide  students  with  the  opportunity  to  set  plans  to  reach  their  goals 
External  Provide  students  with  a challenging  topic  to  solve  and  at  the  same  time  provide 
students  with  the  necessary  support  to  address  the  challenge  (e.g.,  allowing 
collaboration,  giving  time  to  think  or  providing  supportive  materials  if  necessary) 
Instead  of  correcting  a student’s  errors,  ask  students  to  think  what  could  be  done  to 
solve  the  problems 

Raise  a question  that  would  help  students  to  solve  a problem  or  generate  an 
understanding  that  directly  provides  information,  particularly  if  the  acquired 
information  involves  high  levels  of  thought  or  meta-cognition. 

Provides  opportunities  to  demonstrate  competency  and  understanding 
Assessments  Internal  Encourage  self-awareness  and  self-assessment  during  the  learning  process 
External  Challenge  students  without  threatening  their  self-efficacy.  In  online  learning 

environments,  assessment  and  evaluations  should  be  ongoing,  embedded  in  daily 
activities,  focused  on  personal  progress  and  promote  an  understanding  that  errors  are 
opportunities  to  learn. 

Provide  peer  evaluation  opportunities  to  check  the  collaboration  process,  which 
consists  of  shared  meaning-making,  negotiations,  and  building  consensus. 

Have  them  participate  in  decision  making  processes  needed  to  learn  (e.g.,  choosing 
learning  strategies,  project  topic,  type  of  information  necessary  and  selecting  a 
collaborator) 

Feedback  Internal  Provide  students  opportunities  to  review  and  reflect  on  themselves  as  learners 

External  Provide  feedback  on  students’  reflection 

Emphasize  students’  attempts  and  students’  initiation 

Provide  supportive  emotional  feedback 

Emphasize  the  importance  of  errors  as  a process  of  learning 

Provide  encouragement  and  praise  students’  efforts  spent  on  solving  problems 

Provide  helpful  but  not  critical  tones 

Interactions  Internal  Provide  students  with  opportunities  to  self-talk  and  reflect  about  their  learning 

External  Raise  a controversial  issue  for  interaction  so  that  learners  compare  and  contrast 
learning  strategies  using  meta-cognition,  cognition  and  reflection 
Have  students  recognize  that  interaction  is  a process  of  shared  meaning-making, 
negotiating  and  building  consensus  by  explaining  and  modeling  it 
Encourage  interactions  among  peers  and  between  students  and  teachers  by  making  the 
learning  environments  warm  and  nice. 

Let  students  know  there  is  no  stupid  question  and  every  question  is  valuable  not  only 
to  themselves  but  also  to  all  the  students 

We  argue  that  scaffolding  should  be  offered  in  the  mediator  part  of  our  model,  which  is  the  context  in  this 
case.  The  context  includes  the  tasks,  assessment,  feedback  and  interaction.  We  excluded  technology  because  it  is  the 
tool  used  to  deliver  the  scaffolding  strategies.  Under  the  context,  scaffolding  students’  self-regulation  can  be  divided 
into  two  parts:  one  is  for  scaffolding  knowledge  as  learners,  which  is  internal  scaffolding,  and  the  other  is  for 
scaffolding  knowledge  about  self-regulated  learning  strategies,  which  is  external  scaffolding.  We  also  suggest  these 
two  types  of  scaffolding  should  be  embedded  into  the  learning  tasks  and  learning  processes  in  online  learning. 

Conclusion 

In  this  paper,  we  first  reviewed  the  self-regulated  learning  literature.  We  proposed  there  are  two  types  of 
self-regulation  processes.  The  first  type  of  self-regulation  is  the  individual  regulation  process  and  the  second  type  is 
the  co-regulation  process  among  a group  of  learners.  In  online  learning  environments,  where  context  mediate  human 
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interaction,  we  argue  self-regulation  should  be  defined  based  on  what  type  of  learning  process  in  which  they  will 
need  to  be  engaged.  Online  learning  demands  students  to  be  independent  learners  through  individual  regulation.  At 
the  same  time,  it  demands  students  to  be  good  negotiators  and  collaborators  through  co-regulation  with  others. 
Therefore,  the  definition  of  self-regulation  in  online  learning  environments  should  cover  individual  and  co- 
regulation. Based  on  these  assumptions,  we  define  self-regulated  learners  in  online  environments  as  those  who  have 
knowledge  about  themselves  as  a learner,  independent  learners  and/or  collaborators.  Furthermore,  students’  self- 
regulation can  be  supported  by  scaffolding  and  instructional  designers  and  instructors  should  consider  the  context 
when  they  scaffold  student  self-regulation  in  online  learning. 
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Abstract 

Students  bring  misconceptions  about  evolution  theory  to  the  classroom  before  they  learn  the  offered 
content.  These  misconceptions  are  not  easily  replaced  with  the  correct  conceptions  even  after  a semester  course. 
Many  researchers  argued  teaching  students  about  nature  of  science  would  contribute  to  changing  their 
misconceptions.  However,  little  empirical  research  has  been  done.  We  investigate  the  effects  of  teaching  nature  of 
science  on  students'  conceptual  change.  The  results  show  that  teaching  nature  of  science  in  the  class  does  not 
influence  students'  conceptual  change.  Implications  of  this  research  are  discussed. 

Introduction 

Learning  is  a process  of  conceptual  change  (Posner,  Strike,  Hewson,  & Gertzog,  1982).  Ideally,  conceptual 
change  occurs  through  learner’s  proactive  revision  and  reorganization  of  the  preexisting  knowledge.  In  this  case, 
learners’  preexisting  knowledge  is  helpful  to  promote  conceptual  change.  However,  in  science  education,  students’ 
preexisting  conceptions  often  prohibit  students’  conceptual  change.  In  many  cases,  students’  preexisting  conceptions 
are  recognized  as  misconceptions. 

Misconceptions  are  defined  as  the  concepts  that  have  particular  interpretations  and  meaning  in  students’ 
articulations  that  are  not  scientifically  accurate  (Bahar,  2003).  Common  examples  of  student  misconceptions 
include:  1)  belief  that  species  evolve  because  of  their  own  “needs”,  2)  belief  that  “use”  or  “misuse”  of  some 
functions  determines  the  form  of  species,  3)  belief  that  the  environments  influence  species  evolution  in  short  time 
periods  (Bishop  & Anderson,  1990),  and  4)  belief  that  Lamarckian  concepts  are  appropriate  to  explain  evolution 
theory  (Cunnigham  & Wescott,  2003).  Students’  misconceptions  are  the  results  of  their  intuitions,  everyday 
experience,  and  cultural  backgrounds  (Vosniadou,  1992).  Although  students’  misconceptions  are  naive,  they  explain 
their  misconceptions  with  coherent  mechanism.  That’ s why  misconceptions  are  hard  to  change  even  after  the 
instruction.  In  particular,  students’  misconceptions  about  evolution  theory  in  biology  class  are  not  easily  changed. 

Many  researchers  (Scharmann  & Harris,  1991;  Qian  & Alvermann,  2000)  argue  students’  beliefs  about  the 
nature  of  science  (NOS)  are  a critical  factor  to  promote  the  students’  conceptual  change.  Qian  and  Alvermann 
(1995)  found  the  high  school  students  who  have  immature  beliefs  on  the  nature  of  science  think  that  scientific  facts 
are  a product  of  an  empirical  process  and  science  exists  to  make  human  life  more  convenient  through  technology. 
Due  to  these  immature  beliefs  on  the  nature  of  science,  the  students  don’t  understand  the  multiplicity  and 
complexity  of  the  scientific  disputes.  Also,  this  rigidity  of  the  beliefs  on  the  nature  of  science  hinders  them  from 
changing  their  misconceptions  (Qian  & Alvermann,  2000). 

Researchers  (Demastes,  Good,  & Peebles,  1995;  Pintrich,  Marx,  & Boyle,  1993;  Sharmann  & Harris,  1991; 
Qian  & Alvermann,  2000)  argue  teaching  nature  of  science  would  contribute  to  conceptual  change.  Scharmann  and 
Harris  (1991)  argued  that  understanding  of  the  nature  of  science  will  promote  the  students’  conceptual  change. 
Although  many  researchers  argued  the  importance  of  teaching  nature  of  science,  there  is  little  empirical  research  to 
support  their  arguments.  The  primary  purpose  of  present  research  is  to  investigate  the  effects  of  teaching  nature  of 
science  on  students’  conceptual  change.  The  second  purpose  is  to  identify  what  misconceptions  non-biology-major 
college  students  have.  More  specifically,  our  research  questions  are 

1 . Is  teaching  students  about  nature  of  science  effective  for  conceptual  change  in  natural  selection? 

2.  What  misconceptions  college  students  bring  to  the  class? 

3.  What  misconceptions  have  not  been  changed  after  the  instruction? 
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Methodology 


Participants 

The  participants  for  this  research  were  undergraduate  students  who  enrolled  in  Introduction  to  Biological 
Anthropology  in  a mid- west  university.  126  students  enrolled  the  class  and  most  of  them  are  non-major  students. 
Among  them,  37  students  volunteered.  28  students  were  female  and  9 were  male.  Also,  34  students’  were  between 
18  to  21  years  of  age.  Ten  students  were  freshmen,  20  students  were  sophomores,  two  students  were  juniors  and  five 
students  were  seniors.  Twelve  students  took  one  biology  course  in  high  school  or  college  and  12,  6,  and  2 students 
took  two,  three,  and  over  four  biology  classes,  respectively.  Five  students  had  never  taken  a biology  class  before  this 
class. 

Learning  Context 

“Introduction  to  Biological  Anthropology”  fulfils  a university  biological  science  credit  requirement,  and  a 
math  reasoning  requirement  if  taken  in  conjuction  with  a laboratory  course.  Attendance  in  a class  is  not  required,  but 
attendance  in  the  laboratory  is  required.  The  instructor  gave  one  hour  lecture  three  times  a week,  and  note 
summaries  were  available  through  the  WebCT  course  management  system.  The  curriculum  covered  nature  of 
science,  development  of  evolution  theory,  forces  of  evolution,  taxonomy  and  primates,  and  paleontology. 

Instrument 

In  order  to  identify  students’  conceptual  change,  a quantitative  survey  questionnaire  was  distributed  before 
and  after  students  learn  natural  selection.  The  questionnaire  was  developed  by  Anderson,  Fisher,  and  Norman 
(2002).  The  survey  consists  of  20  items  and  asks  about  important  conceptions  of  natural  selection  such  as  biotic 
potential,  variation  within  a population,  variation  inheritable,  differential  survival,  origin  of  species,  population 
stability,  and  so  on. 

Procedures 

Before  students  learned  natural  selection  part,  which  is  the  fourth  week  of  the  class,  we  visited  the 
classroom  and  explained  the  purpose  and  importance  of  this  research.  We  distributed  Anderson,  Fisher,  and 
Norman’s  (2002)  conceptual  inventory  of  natural  selection  to  students  as  a pre-test.  Seven  weeks  after  students 
learned  natural  selection,  we  administered  the  conceptual  inventory  of  natural  selection  again  as  a post-test. 

Teaching  nature  of  science 

The  instructor  provided  students  an  orientation  about  what  science  is,  how  theory  is  developed,  and  the 
relationship  between  science  and  religion  for  a week.  In  addition,  he  reminded  students  what  science  is,  how  theory 
developments  can  be  applied  to  understand  natural  selection  and  evolution  theory  during  the  semester.  His  teaching 
is  based  on  revision  of  Firenze  (1997)’ s suggestions.  Firenze  suggested  five  process  of  introducing  evolution  theory 
to  students;  engage,  explore,  explain,  elaborate,  and  evaluate.  In  this  class,  the  instructor  revised  the  Firenze 
(1997)’s  suggestions  as  follows: 

1.  Engage:  introduce  some  morphological,  physiological,  and/or  behavioral  adaptation. 

2.  Explore:  form  discussion  groups  and  prompt  the  students  with  questions  that  are  related  to  their 

misconceptions. 

3.  Explain:  have  the  groups  present  their  ideas  and  encourage  students  to  provide  constructivist  critiques  with 

each  other. 

4.  Elaborate:  provide  feedback  on  the  progress  and  effectiveness  of  the  learning  activity. 

5.  Evaluate:  provide  each  student  with  similar  problem  and  ask  them  to  solve  it. 

Results 

Repeated  Measures  ANOVA  was  conducted  to  identify  the  mean  difference  between  the  pre  and  post  test 
after  the  instruction.  There  was  no  significant  difference  between  the  pre  and  post  test,  t (36)  = -.1.636,  p > 0.5.  This 
means  introducing  NOS  does  not  statistically  effect  conceptual  change.  However,  students’  post  test  results 
(M=15.72,  SD=3.01 1)  was  slightly  higher  than  the  pre  test  results  (M=15.25,  SD=3.1 1 1). 

Using  Anderson,  Fisher,  and  Norman’s  (2002)’ s instrument,  we  analyzed  what  misconceptions  students 
had  before  they  learn  the  content,  and  what  misconceptions  had  not  been  changed  after  the  class.  The  summarization 
is  presented  in  Table  1.  The  frequencies  of  checking  alternative  concepts,  which  are  misconceptions,  are  presented 
in  pre  and  post  test  columns.  Misconceptions  are  presented  with  the  questionnaire  items.  Remember  the  number  in 
the  cell  does  not  mean  the  number  of  students  but  the  frequencies  of  students’  having  the  misconceptions. 
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The  pre-test  column  in  Table  1 shows  that  non-biology-major  college  students  in  Introduction  to  Biological 
Anthropology  do  not  have  as  many  misconceptions  about  natural  selection  as  expected.  The  misconceptions  they 
had  before  they  learn  the  content  were  “populations  level  off’  in  biotic  potential;  “Fitness  is  equated  with  strength, 
speed,  intelligence  or  longevity”  in  differential  survival;  “Changes  in  a population  occur  through  a gradual  change  in 
all  members  of  a population”  in  Change  in  a Population  in  change  in  a population;  “Mutations  occurs  to  meet  the  needs 
of  the  population;”  “Organisms  can  intentionally  becomes  new  species  over  time  (an  organism  tries,  wants,  or  needs 
to  become  a new  species)”  in  origin  of  a species  and  “Mutations  are  intentional:  an  organism  tries,  needs,  or  wants  to 
change  genetically”.  Post  test  column  in  Table  1 shows  what  misconceptions  have  not  been  changed  after  the  class. 
The  only  misconception  to  significantly  change  after  instruction  is  “changes  in  a population  occur  through  a gradual 
change  in  all  members  of  a population”. 


Table  1.  Students’  Misconceptions  in  Natural  Selection  before  and  after  the  Class 


Topic 

Alternative  Conceptions 

Pre 

Post 

Biotic  Potential 

Not  all  organisms  can  achieve  exponential  population  growth  (11C) 

0 

0 

Organisms  only  replace  themselves.  (1A,  11  A) 

1 

0 

Populations  level  off  (IB,  11D,  ID) 

18 

17 

Population  Stability 

All  populations  grow  in  size  over  time  (3 A,  12B) 

0 

0 

Populations  decrease  (3D,  12C). 

1 

0 

Populations  always  fluctuate  widely/  randomly  (3C,  12D). 

2 

5 

Natural  Resources 

Organisms  can  always  obtain  what  they  need  to  survive  (2B,  2C,  2D,  14A,  14B,  14C) 

7 

8 

Limited  Survival 

There  is  often  physical  fighting  among  one  species  (or  among  different  species)  and 
the  strongest  ones  win  (5B,  15B). 

3 

1 

Organisms  work  together  (cooperate)  and  don’t  compete  (5 A,  5C,  15 A). 

2 

3 

Variation  Within  a 
Population 

All  members  of  a population  are  nearly  identical  (9 A,  16A). 

0 

1 

Variations  only  affect  outward  appearance,  don’t  influence  survival  (9B,  9C,  16B). 

7 

7 

Organisms  in  a population  share  no  characteristics  with  others  (16D). 

1 

0 

Variation 

Inheritable 

When  a trait  (organ)  is  no  longer  beneficial  for  survival,  the  offspring  will  not  inherit 
the  trait  (7B,  7B). 

5 

7 

Traits  acquired  during  an  organism’s  lifetime  will  be  inherited  by  offspring.  (7 A, 
17A) 

0 

0 

Traits  that  are  positively  influenced  by  the  environment  will  be  inherited  by  offspring 
(7D). 

3 

3 

Differential 

Survival 

Fitness  is  equated  with  strength,  speed,  intelligence  or  longevity  (10A,  10B,  18A, 
18C,  18D). 

16 

12 

Organisms  with  many  mates  are  biologically  fit  (10D). 

7 

1 

Change  in  a 
Population 

Changes  in  a population  occur  through  a gradual  change  in  all  members  of  a 
population  (4A,  13 A,  7C). 

22 

9 

Learned  behaviors  are  inherited  (4C,  13C). 

0 

4 

Mutations  occurs  to  meet  the  needs  of  the  population  (4D,  13D) 

17 

11 

Origin  of  a Species 

Organisms  can  intentionally  become  new  species  over  time  (an  organism  tries,  wants, 
or  needs  to  become  a new  species)  (8C,  8D,  20A,  20D) 

22 

19 

Speciation  is  a hypothetical  idea  (8B,  20C). 

1 

1 

Origin  of  the 
Variation 

Mutations  are  adaptive  responses  to  specific  environmental  agents  (6C,  15C,  19D). 

7 

6 

Mutations  are  intentional:  an  organism  tries,  needs,  or  wants  to  change  genetically 
(6A,  6D,  19 A,  19B). 

23 

25 

Discussion 

Our  research  results  show  that  teaching  college  students  nature  of  science  does  not  statistically  alter 
conceptual  change.  There  could  be  several  reasons.  First,  the  duration  of  NOS  teaching  is  too  short.  That  is,  one 
week  teaching  of  natural  of  science  does  not  influence  students’  understanding  of  science.  Considering  students’ 
misconceptions  are  results  of  their  everyday  life  experience,  cultural  background,  and  community  philosophy  (Qian 
& Averman,  2000),  one  week  instruction  may  not  influence  student’s  understanding  of  nature  of  science.  Second, 
students  did  not  apply  the  nature  of  science  into  their  learning  context,  which  is  natural  selection.  Knowing  and 
applying  are  different  issues.  Southerland  and  Sinatra  (2003)  distinguished  understanding  and  accepting.  According 
to  them,  if  the  task  is  controversial  or  complex,  learners’  acceptance  and  learners’  understanding  are  not  related.  In 
this  case,  students’  intentional  efforts  are  more  necessary  for  their  conceptual  change.  Learning  natural  selection  is  a 
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complex  and  can  be  controversial  issue  depending  on  persons.  Therefore,  learning  natural  selection  requires  students 
to  actively  participate  in  learning  process  with  high  motivation  and  appropriate  deep  process  of  learning  strategies 
(Printrich,  Marx,  & Boyle,  1993).  Last,  students  may  not  understand  the  importance  of  learning  nature  of  science.  If 
students  do  not  value  learning  nature  of  science,  they  will  not  deep  process  and  pay  less  effort  to  learn  natural 
selection  (Pintrich,  Marx,  & Boyer,  1993).  Furthermore,  they  will  not  apply  the  nature  of  science  into  their  learning 
context  (Jonassen,  Marra,  & Palmer,  2002). 

In  this  research,  the  instrument  results  showed  that  students’  do  not  have  many  misconception  about  natural 
selection  compared  to  previous  research  (Cunningham  & Wescott,  2003).  Students’  pre  test  mean  was  15.25. 
Cunningham  and  Wescott  (2003)  reported  the  misconceptions  over  50%  of  the  college  students  among  229  have  the 
misconception  that  species’  characteristics  gradually  change  over  time  and  that  individuals  evolve  because  they  need 
to.  However,  in  our  research,  we  were  not  able  to  find  those  high  rates  of  misconceptions.  Most  of  students’ 
misconceptions  about  natural  selection  in  this  research  were  at  relatively  low  rates. 

Interestingly,  the  misconceptions  we  observed  were  identified  in  previous  research  (Bishop  & Anderson, 
1990).  At  this  point,  we  raise  the  questions,  why  students  have  similar  misconceptions  over  time  and  why  these 
misconceptions  are  not  changed  even  after  the  class?  Does  this  mean  our  educational  system  has  not  been  changed 
over  time  even  though  a lot  of  different  instructional  strategies  (e.g.,  constructivists  learning  environments,  goal- 
based  scenario,  situated  learning,  and  so  on)  and  new  technologies  are  integrated  in  science  education?  Discussing 
about  this  issue  will  be  endless. 

Last,  our  research  found  that  students’  misconceptions  in  natural  selection  are  not  easily  changed  even  after 
the  class.  The  results  are  in  line  with  other  research  results  (Bishop  & Anderson,  1990).  We  found  only  one  concept 
showed  statistically  significant  change;  “Changes  in  a population  occur  through  a gradual  change  in  all  members  of 
a population”.  Then,  what  is  the  difference  between  the  concept  changed  and  concepts  having  not  been  changed? 
More  qualitative  research  should  be  conducted  to  identify  the  difference  between  two  kinds  of  concepts.  If  we  found 
the  difference  between  two  concepts,  it  will  contribute  to  teaching  natural  selection  and  evolution  theory. 
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Abstract 

The  Minnesota  Educators  and  Leaders  Technology  Initiative  ( MELTI ) contributes  to  technology 
integration  and  digital  equity  through  computer- supported  collaborative  learning  activities  and  other  project 
activities.  The  computer- supported  collaborative  learning  ( CSCL)  activities  that  include  asynchronous, 
synchronous,  and  video  conferencing  are  designed  to  address  issues  of  integration  faced  by  the  MELTI  participating 
schools.  The  notions  of  social  inclusion  and  technology  literacy  play  a key  role  in  the  CSCL  activities  design  for  the 
purpose  of  overcoming  barriers  to  digital  equity. 


Introduction 

Traditionally,  digital  divide  is  perceived  in  terms  of  a gap  between  the  ‘haves”  and  have-nots.”  However, 
recent  discussions  have  shifted  the  focus  to  social  inclusion  as  the  key  to  bridging  the  digital  divide  (Cisler,  2000; 
Commission  of  the  European  Communities,  2001;  Warschauer,  2002).  Social  inclusion  is  defined  as  the  “process  by 
which  efforts  are  made  to  ensure  that  everyone,  regardless  of  their  experiences  and  circumstances,  can  achieve  their 
potential  in  life.  To  achieve  inclusion  income  and  employment  are  necessary  but  not  sufficient.  An  inclusive  society 
is  also  characterized  by  a striving  for  reduced  inequality,  a balance  between  individuals’  rights  and  duties  and 
increased  social  cohesion.”  (Centre  for  Economic  & Social  Inclusion,  2002).  With  98%  of  K-12  schools  in  the 
United  States  wired  to  the  Internet  (Advanced  Learning  Technologies,  2002),  access  to  hardware  is  no  longer  the 
issue.  Access  to  technology  integration  into  curriculum  and  technology  literacy  has  become  the  top  priority  in 
overcoming  the  barriers  to  digital  equity  in  the  K-12  system.  Educators  need  to  be  empowered  with  the  knowledge, 
skills,  and  institutional  support  in  technology  integration  in  order  to  help  bridge  the  gap.  Policy  makers  and  school 
leaders  need  to  provide  the  provisions  and  institutional  support  for  a more  comprehensive  plan  to  support  the  cause. 
This  paper  introduces  how  in-service  teachers  (i.e.,  licensed  teachers  currently  teaching  in  the  classroom)  work 
together  to  bridge  the  divide  in  the  K-12  classrooms  through  computer  conferencing  and  face-to-face  training.  The 
Minnesota  Educators  and  Leaders  Technology  Initiative  (MELTI)  aims  to  enhance  K-12  educators’  technology 
competency  and  establish  cadres  of  collaborative  teaching  partnership  for  technology  integration.  Computer- 
supported  collaborative  learning  (CSCL)  activities  that  play  a key  role  in  the  MELTI  project  will  be  described  in  this 
paper. 


Literature  Review 

In  the  digital  age,  it  is  no  longer  enough  to  provide  hardware  and  software  to  educators.  Educators  need 
professional  training  in  the  use  of  information  and  communication  technology  (ICT)  and  empowerment  in  the  best 
practices  in  integrating  technology  into  the  curriculum.  Furthermore,  Warschauer  (2002)  contends  that  the 
traditional  notion  of  a digital  divide  as  the  bipolar  disparity  in  societal  resources  provides  an  oversimplified  and  poor 
roadmap  for  utilizing  technology  to  bridge  the  gap.  He  maintains  that  there  is  no  binary  divide  and  social  inclusion 
is  the  key  in  dealing  with  the  complex  issue.  He  advocates  a model  of  technology  for  social  inclusion  that  takes  into 
account  physical,  digital,  human  and  social  resources  to  promote  effective  use  of  ICTs.  These  four  areas  of  resources 
are  defined  in  the  following  terms: 

Physical  resources  encompass  access  to  computers  and  telecommunication  connections.  Digital  resources 
refer  to  digital  material  that  is  made  available  online.  Human  resources  revolve  around  issues  such  as  literacy  and 
education  (including  the  particular  types  of  literacy  practices  that  are  required  for  computer  use  and  online 
communication).  Social  resources  refer  to  the  community,  institutional,  and  societal  structures  that  support  access  to 
ICT”  (Warschauer,  2002). 

This  model  presents  a comprehensive  approach  to  promote  digital  equity  that  involves  every  sector  of  the 
society.  The  current  practice  has  put  much  emphasis  on  providing  physical  and  digital  resources.  Policy  makers, 
school  leaders,  educators,  and  the  community  are  equal  contributors  to  the  effective  use  of  ICTs  and  need  to  be  part 
of  the  overall  planning  of  ICT  integration. 

In  examining  the  barriers  to  digital  equity  in  education,  Fulton  & Sibley  (2003)  have  focused  on  the  issue 
of  access  to  the  following  resources: 

• Access  to  up-to-date  hardware,  software,  and  connectivity 

• Access  to  meaningful,  high-quality,  and  culturally  responsive  content  along  with  the  opportunity  to 
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contribute  to  the  knowledge  base  represented  in  online  content 

• Access  to  educators  who  know  how  to  use  digital  tools  and  resources  effectively 

• Access  to  systems  sustained  by  leaders  with  vision  and  support  for  change  through  technology  (p.  15) 
Educators  need  to  have  access  to  high-performance  technology  so  that  they  can  fully  utilize  various  digital 

resources  and  applications  effortlessly.  Researchers  have  found  that  22  million  Americans  who  live  below 
$ 14,500/year  lack  the  literacy  skills  to  use  the  Internet  (Lazarus  & Lipper,  2000).  Without  increasing  the  literacy 
level,  more  Americans  will  be  ignored  and  continue  without  the  benefits  of  what  Internet  resources  can  offer  them. 
Furthermore,  educators  need  to  have  access  to  innovative  pedagogy  and  good  examples  of  teaching  with  digital 
resources.  While  an  increasing  number  of  school  districts  have  recognized  the  importance  of  providing  professional 
training  opportunity  for  teachers,  school  leaders  also  need  to  be  more  active  in  participating  in  the  process  to  support 
the  change. 

The  model  of  technology  for  social  inclusion  and  discussion  regarding  barriers  to  digital  equity  have 
shaped  the  theoretical  framework  in  the  design  of  computer-supported  collaborative  learning  activities  mentioned  in 
this  paper.  CSCL  systems  are  powerful  media  for  professional  development  in  teacher  education.  CSCL 
environments  can  encourage  conversation  (Jenlink  & Carr,  1996),  reflection,  idea  sharing,  interaction,  knowledge 
construction  (Scardamalia  & Bereiter,  1996),  and  community  building  (Pea,  1993).  CSCL  has  become  an  integral 
part  of  teacher  education.  The  following  sections  will  describe  how  these  theories  are  infused  into  the  design  of 
CSCL  activities  in  the  MELTI  project. 


Background 

The  MELTI  project  is  a one-year  grant  project  that  is  supported  by  the  E2T2  Grant  from  the  Minnesota 
Department  of  Education.  Two  strands  of  training  for  both  K-12  school  leaders  and  teachers  are  provided.  This 
paper  will  focus  on  the  training  for  K-12  teachers.  The  MELTI  project  consists  of  sixteen  charter  schools,  two  public 
schools,  and  two  private  schools.  Charter  schools  are  independent  schools  designed  and  operated  by  educators, 
parents,  community  leaders,  entrepreneurs  and  others.  Charter  schools  are  exempt  from  many  state  and  local 
regulations.  The  member  schools  have  a high  number  of  minority  students,  including  Hmong  (Asian),  African,  and 
Hispanic  Americans.  School  educators  who  participate  in  the  initiative  will  complete  a two-part  professional 
development  program  in  technology  integration.  The  first  part  consists  of  a four-course  twelve-credit  graduate 
certificate  program  in  learning  technology  at  the  University  of  St.  Thomas.  The  second  part  features  a capstone 
seminar  in  which  participants  showcase  the  best  practices  of  technology  integration  and  share  experience  in  the 
implementation  of  technology-enriched  curriculum. 

The  participants  in  the  Learning  Technology  Program  explore  ways  to  (1)  improve  classroom  learning 
through  the  application  of  innovative  technologies  to  instruction  and  curriculum  development,  (2)  understand, 
demonstrate  and  integrate  essential  educational  technology  tools,  and  (3)  become  leaders  in  educational  change.  As 
a result  of  the  graduate  certificate  program,  graduates  are  able  to  communicate,  design,  and  implement  innovations 
in  their  classrooms  and  communities. 

Sixty  educators  receive  free  training  and  participate  in  the  Learning  Technology  Program.  There  are  three 
cohorts  of  twenty  students  each,  two  in  the  Metro  Twin  Cities  area  and  one  in  Duluth  (200  miles  from  the  Twin 
Cities).  They  will  receive  a graduate  certificate  of  Learning  Technology  when  they  complete  the  four  courses  in  the 
program  in  the  span  of  one  year.  The  graduate  program  features  the  following  courses:  Introduction  to  Learning 
Technology,  Internet  Web  Site  and  Educational  Software  Evaluation,  Design  and  Development  of  Digital  Media, 
and  Integration  of  Technology  into  the  Classroom.  Each  course  addresses  the  International  Society  for  Technology 
in  Education  (ISTE)  National  Educational  Technology  Standards  (NETS)  for  teachers.  All  of  the  courses  are 
designed  based  on  the  adult  learning  model  and  on  project-based  learning.  The  participants  will  devise  projects  with 
practical  and  direct  application  in  the  classroom. 

The  central  themes  woven  into  the  program  are  the  design  of  an  e-Folio  to  support  professional 
development,  the  employment  of  data-driven  decision-making  tools  for  student  performance  analysis,  and  the 
integration  of  technology  into  the  curriculum.  To  fulfill  the  requirements  for  each  course,  participants  will  present  an 
integration  project  using  a technology  tool  that  is  to  be  directly  applied  in  the  classroom  or  community. 

Participants  will  have  access  to  several  Web-based  authoring  tools,  tutorials,  and  conferencing  systems.  In 
addition  to  Blackboard  as  the  main  course  management  system  and  conferencing  medium,  video-conferencing  is 
also  utilized  for  guest  speakers  and  virtual  tours.  This  initiative  will  provide  the  hardware  and  software  needed  for 
the  participants  to  ensure  that  there  is  no  digital  disconnect  between  professional  development  and  classroom 
practice.  Participants  will  have  the  support  of  tech  mentors  and  program  advisors  during  the  implementation  process. 
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CSCL  Activities 

Several  CSCL  activities  are  built  in  all  four  courses.  Cohort  members  are  encouraged  to  collaborate  on 
various  projects  throughout  the  program.  The  activities  are  designed  to  address  the  two  themes  of  the  MELTI 
projects:  technology  integration  and  digital  equity. 

CSCL  for  technology  integration 

Throughout  the  program,  participants  meet  on  a regular  basis  for  professional  development  and  stay 
connected  through  asynchronous  communication  in  Blackboard.  Several  projects  were  designed  to  assist  them  with 
technology  integration  in  the  classroom.  As  of  the  writing  of  this  paper,  the  first  course  has  just  begun.  Formative 
and  summative  evaluation  will  be  conducted  during  the  course  and  at  the  end  of  the  project.  Here  is  a summary  of 
the  projects  that  will  be  carried  out  through  the  Blackboard  Discussion  Board  and  other  CSCL  systems: 

1.  Project  gallery:  Each  student  has  to  present  an  electronic  portfolio  with  artifacts  that  address  the  ISTE 
standards  to  demonstrate  their  competency  and  progress  in  using  technology.  The  artifact  refers  to  an 
integration  project  developed  in  each  course.  It  can  be  a PowerPoint  presentation  or  a Web-based  lesson. 
They  will  be  adding  new  artifacts  at  the  end  of  each  course.  The  eFolio  Minnesota,  a free  web-based 
portfolio  authoring  system  for  Minnesotans,  is  used  for  the  participants  to  present  the  artifacts  for  the 
portfolio.  The  participants  can  choose  to  present  either  a teaching  portfolio  or  a professional  portfolio  or 
both.  The  eFolio  addresses  are  posted  in  Blackboard  for  peer  review.  Each  participant  will  have  to  review 
at  least  two  other  eFolios  and  provide  feedback. 

2.  Community  building:  In  addition  to  the  regular  Blackboard  course  web  site  for  each  class  in  every  cohort,  a 
community  Blackboard  site  is  set  up  to  provide  a common  space  for  members  of  different  cohorts  to  share 
ideas  and  projects.  Normally,  participants  would  only  review  and  critique  the  projects  from  members  of  the 
same  cohort.  By  creating  a community  site,  they  will  have  a chance  to  see  all  three  cohort  members’ 
accomplishments  and  learn  from  each  other. 

3.  Video-conferencing:  With  the  support  of  the  grant,  Web  Cams  are  purchased  to  support  collaboration 
among  the  cohort  members.  Teachers  can  use  the  video  cameras  in  their  classrooms  for  cross-school 
collaboration.  For  example,  a 10th  grade  teacher  can  assign  his  or  her  students  as  the  buddies  for  the  third 
grade  students  of  another  cohort  member  in  another  school  through  video  conferencing  to  explain  a concept 
or  present  a project.  The  teachers  are  also  encouraged  to  find  international  partners  in  other  part  of  the 
globe  for  cross-cultural  collaboration. 

4.  Tappedln  group  conferencing:  Teachers  are  introduced  to  Tappedln  (http://tappedin.org),  an  “online 
workplace  of  an  international  community  of  education  professionals.”  They  can  create  an  office  space  and 
specialized  groups  for  class  meetings.  The  Tappedln  environment  has  both  synchronous  and  asynchronous 
components  which  are  perfect  for  schools  that  do  not  have  their  own  course  management  systems.  The 
teachers  can  bring  their  students  to  meet  with  a class  from  another  state  to  discuss  specific  issues  such  as 
weather  or  water  conservation  across  different  cultures. 

CSCL  for  Digital  Equity 

Many  of  the  MELTI  activities  are  designed  based  on  Fulton  and  Sibley’s  (2003)  framework  of  the  barriers 
to  the  educational  equity  in  a digital  age.  With  the  grant  support,  each  participant  is  provided  with  a pen  drive  for 
data  storage  and  coupons  for  purchasing  software.  Forty  laptops  are  purchased  and  set  up  to  provide  in-class 
instruction.  At  the  end  of  the  cohort,  the  laptops  will  be  dismantled  and  given  to  all  participating  schools.  In  addition 
to  the  professional  development  opportunity  with  the  Learning  Technology  Program,  onsite  coaching  will  be 
provided  to  all  schools  on  special  topics.  The  grant  will  pay  for  experts  to  conduct  workshops  on  a specialized  topic 
about  which  the  teachers  from  the  same  school  would  like  to  learn  more.  The  computer  conferencing  activities  are 
also  set  up  for  the  purposes  of  providing  a venue  for  teachers  to  strategize  ways  to  overcome  the  barriers  to  digital 
equity. 

1.  Spotlight  presentation:  The  participants  look  for  ways  to  use  various  technological  or  Internet  resources  to 
enhance  classroom  teaching.  They  will  research  on  the  web  and  present  to  the  class  innovative  ways  of 
employing  technology  that  are  appropriate  for  their  classroom.  The  presentation  and  resources  used  will 
then  be  posted  onto  the  Blackboard  discussion  board  for  follow-up  discussion.  The  diverse  background  of 
the  participants  will  help  to  enhance  the  breadth  and  depth  of  the  presentation.  Some  teachers  have  easy 
access  to  computer  labs  with  a wealth  of  computer  resources.  Some  teachers  are  in  a one-computer 
classroom  and  have  no  access  to  a computer  lab.  They  can  provide  each  other  with  ideas  on  how  to 
maximize  the  learning  process  with  the  minimal  resources  through  computer  conferencing. 

2.  Technology  literacy  through  virtual  training:  As  Fulton  and  Sibley’s  (2003)  research  has  found,  a lack  of 
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time  and  technological  skills  are  no  longer  the  main  barriers  to  digital  equity.  Educational  beliefs  also  shape 
the  types  of  classroom  activities  (e.g.,  teacher-centered  v.s.  student-centered  teaching).  Teachers  need  to 
have  access  to  other  teachers  who  excel  in  using  technology.  Throughout  the  MELTI  projects,  excellent 
technology  educators  will  be  invited  as  guest  speakers  to  meet  with  the  cohort  members  through  computer 
conferencing  systems  such  as  Tappedln  and  video-conferencing  to  discuss  the  pedagogical  applications  of 
various  educational  technology  and  showcase  classroom  projects. 

3.  Technology  integration  clinic:  During  the  course  of  the  MELTI  project,  teachers  will  be  able  to  pose 
questions  on  a specific  technology  integration  issue  and  solicit  responses  from  cohort  instructors  and 
members.  The  technological  backgrounds  of  the  cohort  members  range  from  advanced  users  of  technology 
to  novice  users  who  have  little  exposure  to  technology  applications  before  joining  the  cohort.  The  clinic 
will  provide  a venue  for  teachers  who  need  more  support  to  ask  questions.  Participants  who  provide  the 
most  answers  will  be  recognized  at  the  capstone  seminar  at  the  end  of  the  project. 

Conclusion 

The  long-term  impact  of  the  MELTI  project  has  yet  to  be  assessed.  Formative  evaluation  is  underway  and 
summative  evaluation  will  be  conducted  at  the  end  of  the  project.  In  the  evaluation,  the  emphases  will  be  placed  on 
the  teachers’  experience  in  all  of  the  CSCL  activities.  What  has  worked  and  what  hasn’t  worked  for  them?  What  are 
the  obstacles  or  contributing  factors  to  technology  integration  and  digital  equity?  The  MELTI  project  strives  to 
provide  the  member  schools  and  K-12  educators  with  physical  resources  (e.g.,  the  hardware  and  software),  digital 
resources  (e.g.,  relevant  technology  integration  lesson  plans  on  the  Web),  human  resources  (e.g.,  technology  literacy 
education),  and  social  resources  (e.g.,  community -based  learning  and  institutional  support).  Through  technology 
enrichment,  teacher  professional  development,  and  leader  training,  the  MELTI  project  aspires  to  minimize  the 
barriers  to  digital  equity.  The  collaborative  and  community-based  model  of  the  MELTI  project  will  provide  K-12 
educators  and  leaders  in  the  consortium  with  the  skills,  knowledge,  and  tools  to  integrate  technology  into  classroom 
teaching. 
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Enhancing  Science  Field  Trips  with  a Digital  Camera 
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Abstract 

Digital  cameras  in  the  science  classroom  can  he  used  as  a tool  to  enhance  field  trips.  Field  trips  are  meant 
to  provide  students  with  important  hands-on  experiences  to  further  the  goals  and  objectives  of  the  curriculum.  Many 
times  they  turn  into  no  more  than  a day  away  from  school.  These  trips  can  add  value  to  the  curriculum  by  providing 
pre-trip  preparation,  structured  experiences  on  the  trip,  and  experiences  taken  back  to  the  classroom  from  the  trip. 
The  digital  camera  can  be  used  to  develop  materials  to  use  in  preparation  for  the  trip  and  record  aspects  of  the  trip 
to  bring  back  to  the  classroom  for  further  study.  Although  film  based  technologies  can  provide  the  same 
opportunities,  the  use  of  the  digital  camera  provides  a hassle-free  way  to  capture,  manipulate,  and  present  images 
relative  to  the  instruction. 

Field  trips  are  designed  to  provide  students  with  important  hands-on  experiences  to  further  the  goals  and 
objectives  of  the  curriculum.  Many  times  these  trips  turn  into  no  more  than  a day  away  from  school.  In  order  to  add 
value  to  the  curriculum,  field  trips  should  provide  valuable  content  information  to  connect  what  is  being  studied  in 
the  classroom  with  the  out  of  school  world.  Teachers  must  carefully  plan  the  experience  prior  to  the  trip  to  include 
relevant  content  and  methodologies  related  to  the  instructional  experience.  There  are  three  distinct  stages  of  a well- 
planned  field  trip.  First  is  the  pre-trip  preparation  of  the  students,  second  the  experience  of  the  trip  and  thirdly  the 
use  of  the  trip  experiences  in  subsequent  instruction. 

The  pre-trip  preparation  includes  an  overview  of  the  visit  and  tasks  to  be  accomplished  for  further  study 
after  the  trip.  On  the  trip  experiences  should  be  structured  so  that  they  directly  correlate  with  the  experiences  back 
in  the  classroom.  The  camera  can  be  used  to  develop  materials  to  use  in  each  of  these  stages.  Using  a digital 
camera  provides  a relatively  easy  way  to  capture,  manipulate,  and  present  images  to  be  used  in  the  instructional 
process.  Film  based  technologies  can  be  used  to  provide  these  materials  but  is  a much  more  cumbersome  process. 
On  the  field  trip,  incorporating  visual  images  enriches  the  experience  by  allowing  the  students  to  process 
information  visually.  This  limits  the  need  for  students  to  write  and  make  sketches  of  what  was  seen  on  the  trip. 
Photographing  with  a digital  camera  provides  images  that  can  be  viewed  and/or  downloaded  immediately.  Images 
can  be  analyzed  in  the  field  and  if  undesirable  can  either  be  retaken  immediately  or  deleted.  The  ability  to  delete 
unwanted  images  and  to  manage  storage  space  allows  the  user  to  get  the  picture  they  want  and  maximizes  the 
number  of  usable  pictures  taken.  Digital  cameras  save  pictures  in  standard  JPEG  format,  which  is  compatible  with  a 
wide  variety  of  computer  programs  and  can  be  directly  inserted  and  integrated  into  word  processing,  database,  desk- 
top publishing  and  presentation  programs.  Cost  of  image  use,  following  the  initial  outlay  of  funds  for  a digital 
camera  and  storage  device  (flash  card  or  floppy  disk)  is  minimal.  Therefore,  one  time  funding  provides  equipment 
and  materials  for  use  over  several  trips  and  years.  Eliminating  continuous  costs  encourages  teachers  and  student  to 
use  the  camera  frequently. 

Using  the  Camera  as  a Pre-trip  Tool 

Using  the  camera  to  develop  pre-trip  activities  requires  teachers  to  produce  material  to  provide  the  students 
with  an  overview  of  the  coming  experience.  Pictures  of  the  site  to  be  visited  serve  as  an  advanced  organizer  or 
anticipatory  set  to  the  field  trip.  These  materials  can  be  used  to  prepare  students  to  pursue  problem  tasks  in  small 
group  investigation  on  the  trip  by  providing  images  of  actual  examples  of  what  they  will  see.  This  will  help  the 
teacher  explain  the  tasks  to  be  studied  in  the  context  of  the  trip.  Presetting  of  the  task  eliminates  the  need  to  use 
valuable  time  on  the  trip.  As  these  tasks  are  being  set,  they  should  be  integrated  into  the  content  of  the  curriculum 
that  the  field  trip  is  to  complement.  During  instruction,  a lesson  on  the  specific  concepts  that  provide  the 
prerequisite  skills  necessary  for  a task,  or  observation,  could  include  the  images  from  the  location  of  the  trip.  When 
working  on  concept  development  (giving  examples  and  non-examples),  images  from  the  field  trip  location  could  be 
included.  Use  of  a digital  camera  can  provide  images  for  first  observation  opportunities.  Often  times  when  we  first 
observe  something,  we  do  not  know  what  to  look  for.  Using  these  images  the  student  can  note  specifics  and  then  on 
the  trip  they  can  focus  on  these  specifics  and  go  beyond  them. 
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On  the  Field  Trip 

Using  the  camera  on  the  field  trip  provides  the  opportunity  to  preserve  aspects  of  the  experience  that  can 
then  be  brought  back  into  the  classroom  for  further  study.  While  on  the  trip  students  can  document  their 
investigations  by  taking  pictures  of  relevant  experiences.  They  can  gather  images  of  specific  items  that  have  been 
predetermined  to  fit  into  a database  of  information,  and  to  look  for  trends,  patterns  and  functions.  Finally  they  can 
use  the  images  to  record  events,  not  limited  to  those  which  have  been  predetermined,  that  will  be  used  for  future 
study  when  back  in  the  classroom. 

After  the  Field  Trip 

After  the  field  trip  the  student  could  use  the  digital  images  taken  on  the  trip  in  follow  up  classroom  lessons 
to  complete  the  tasks  started  in  the  field.  The  use  of  field  manuals  to  identify  the  specific  species  observed  and  to 
look  for  similarities  and  differences  in  members  of  a population  are  examples  of  this  type  of  task.  Students  can 
develop  presentations  of  their  investigations  in  either  a show  and  tell  experience  at  the  primary  level  or  a report  of 
the  study  that  would  incorporate  both  text  and  images  at  the  middle  grades  and  secondary  school  levels.  On  a field 
trip  designed  to  collect  information  for  further  study  the  images  would  provide  an  important  part  of  the  base  for 
subsequent  instruction.  In  tasks  focusing  on  research  of  specific  topics,  the  images  provide  data  to  either  support  or 
refute  pre-field  trip  hypotheses  and  suggest  other  hypotheses  for  investigation. 

Illustrations  of  Three  Field  Trips 

Three  field  trips  that  can  be  taken  in  the  K-12  situation  will  be  used  to  illustrate  the  use  of  the  digital 
camera  as  an  enhancement  for  the  experience.  At  the  early  primary  level,  field  trips  are  taken  to  explore  the 
student’s  community  and  its  workers;  an  example  of  a field  trip  to  a local  beekeeper  will  be  used.  The  middle 
grades,  where  field  trips  provide  students  with  an  opportunity  to  simulate  activities  done  in  the  world  of  work,  the 
example  will  be  a study  a biological  habitat  of  a coastal  barrier  island.  The  secondary  field  trip,  designed  to  bring 
content  into  the  classroom  to  analyze,  will  investigate  the  integration  of  geometry  into  the  structural  design  of  the 
community  and  its  buildings  following  a visit  to  a city  or  construction  site. 

Primary  Grades 

At  this  level,  social  studies  objectives  focus  on  the  community  and  the  interdependence  of  its  members, 
science  is  working  on  process  skills  such  as  classification  and  observation  and  the  content  objectives  of  the  animal 
life  around  us,  and  math  is  working  on  serration  and  classification.  The  study  of  honey  and  its  production  is  an 
appropriate  topic  and  one  that  will  capture  the  student’ s interest.  The  study  should  include  the  process  and  sequence 
of  honey  production,  the  care  of  the  insects,  and  the  distribution  of  the  products.  A pre-trip  presentation  of  the 
overview  will  familiarize  students  with  what  they  will  see.  There  are  many  things  the  students  can  do  and  see  on  a 
trip  to  the  bee  yards,  including  the  bees.  They  need  to  know  safety  behaviors  to  eliminate  startling  the  bees  and  to 
reduce  the  chance  of  getting  stung.  Young  children  viewing  bees  for  the  first  time  may  become  excited,  crowd 
around  the  bees  to  get  a closer  look  or  run  away  in  fear.  The  beekeeper  will  smoke  the  bees  to  make  them  less 
active.  Children  made  familiar  with  the  purpose  of  smoking  and  what  the  bees  and  the  supers  look  like  through 
digital  images  before  coming  on  the  field  trip  can  remain  calm  and  take  turns  viewing  the  live  bees  without  pushing 
and  shoving,  or  showing  signs  of  fear  that  may  alarm  the  bees.  On  the  trip,  students  will  be  given  a chance  to  view 
the  bees,  the  extraction  of  the  honey  from  the  supers,  the  bottles  of  honey  and  the  wax,  in  addition  to  doing  a little 
research  on  the  site.  One  of  the  research  tasks  for  students  could  be  to  investigate  where  the  bees  gather  their  nectar. 
For  this  task  the  teacher  would  define  survey  areas  by  loops  and  have  the  students  survey  the  bee  activity  within 
each  area.  These  survey  areas  would  include  a variety  of  flower  populations  and  the  students  would  tally  the 
number  of  bees  moving  in  and  out  of  the  area  as  part  of  the  survey.  This  task  requires  students  to  be  able  to  tally 
information.  The  prerequisite  instruction  on  this  could  include  a picture  of  an  area  they  will  see  on  the  field  trip, 
with  bees.  The  students  count  the  bees  in  the  picture  and  make  a tally  mark  for  each  bee  seen.  While  the  concept  to 
be  generalized  is  taught  using  many  different  examples  including  an  example  from  an  upcoming  field  trip  makes  the 
task  on  the  trip  familiar  when  the  students  are  asked  to  do  it. 

At  a primary  level  most  of  what  we  want  the  students  to  do  is  work  on  gathering  broad-based  awareness  of 
the  experience.  On  the  trip  students  could  take  pictures  of  what  they  see  to  help  recall  the  experience  and  set  it  in 
their  knowledge  base.  While  developing  a formal  database  may  not  be  conducive  to  every  field  trip  topic  and  age 
range.  Children  could  take  pictures  of  all  the  types  of  flowers  they  see  the  bees  on  and  use  the  survey  data  begin  to 
sort  by  the  number  of  times  a bee  went  to  each  type  flower.  This  would  provide  a knowledge  base  from  which  to 
discuss  how  different  types  of  honey  are  formed,  i.e.,  orange  blossom,  clover,  and  blackberry.  Pictures  of  anything 
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unusual  that  a child  or  teacher  saw  could  be  taken,  the  children  can  discuss  the  trip  and  determined  more  questions 
to  ask  on  a follow-up  visit  by  the  bee  keeper  to  the  classroom.  The  teacher  and  the  beekeeper  could  take  additional 
pictures  that  may  help  answer  theses  questions  after  the  students  made  the  trip. 

Middle  Grades 

In  middle  grades  science,  students  study  the  knowledge  base  of  the  interdependence  of  the  separate 
populations  in  a specific  habitat.  In  mathematics,  they  use  numbers  to  quantify  trends  and  predict  the  future.  An 
overview  of  the  biological  habitat  of  a coastal  island  provides  the  students  with  a chance  to  study  the  effects  of  the 
different  types  of  land  forms  and  their  plant  life  (the  beach,  primary  dunes,  inter  dune  meadows,  secondary  dunes, 
the  maritime  forest  and  the  marshes).  The  use  of  pre-trip  visuals  encourages  the  students  when  on  the  trip  to  focus 
in  on  more  detail  and  specific  variations  then  a first  time  viewing  would  allow.  A student  who  knows  what  to  look 
for  will  begin  finding  instances  of  the  concept  immediately.  The  teacher  on  this  trip  would  want  the  students  to  not 
only  note  the  different  zones  and  their  habitat  but  to  also  pick  a specific  aspect  to  study,  such  as  the  plant  or  animal 
populations.  On  the  beach,  one  of  the  task  groups  may  be  assigned  to  study  ghost  shrimp  populations.  Photographs 
of  the  area  of  the  beach  where  ghost  shrimp  are  located,  an  active  ghost  shrimp  hole,  and  illustrations  of  appropriate 
survey  techniques  prepare  the  students  to  be  able  to  begin  the  study  upon  reaching  the  beach  without  the  need  for  the 
teacher  to  explain  the  task.  A group  assigned  the  task  of  studying  live  mollusks  on  the  beach,  that  has  the 
prerequisite  knowledge  of  identifying  the  footprint  of  a mollusk,  the  track  the  animal  leaves  and  what  the  animal 
looks  like,  can  begin  to  locate  live  mollusks  immediately.  Unfortunately,  at  any  one  visit  there  is  no  guarantee  that 
specific  shelled  animals  will  be  present  but  from  study  before  the  trip  using  pictures,  the  students  have  trained  their 
eye  so  they  know  what  to  look  for  and  can  transfer  this  knowledge  to  the  animals  present.  On  a trip,  pictures  of 
biological  specimens  of  plants,  shells,  animals,  birds,  and  insects  could  be  taken  and  then  looked  up  in  the  field 
guides  back  in  the  classroom.  Using  a field  guide  with  the  images  after  the  trip  will  encourage  the  students  to  focus 
on  the  details  of  the  specimen  and  spend  time  gathering  more  information  on  the  trip.  Using  the  guides  they  can 
note  likeness  and  differences  in  specific  specimens  within  and  across  species. 

Databases  could  be  developed  of  such  things  as  bird  sighting,  shell  types,  crab  sighting,  etc.  A database 
for  a population  such  as  birds  gives  students  the  chance  to  identify  patterns  in  size  of  populations,  time  of  day  the 
birds  are  seen,  behavior,  and  migratory  patterns.  The  information  in  the  database  could  be  increased  by  the  class  or 
individual  class  members  who  might  return  to  the  site  of  the  field  trip  over  the  year  or  by  using  data  from  previous 
class  trips.  Looking  at  other  pictures  in  the  database  would  lead  to  a discussion  of  the  range  of  the  bird.  For 
instances,  if  a middle  grades  class  kept  the  database  beyond  the  field  trip,  students  could  record  sightings  of  herons 
in  many  different  settings  and  add  it  to  the  database.  Sightings  included  in  the  database  could  be  from  the  Atlantic 
Coast,  Pacific  Coast,  Great  Lakes,  inland  waterways,  and  marshes.  A group  looking  at  live  seashells,  when  doing  a 
population  study,  can  bring  back  information  showing  the  a footprint  of  a buried  mollusk,  the  track  leading  to  the 
animal,  the  uncovered  animal  to  verify  the  type.  The  group  could  then  take  pictures  to  document  the  aspects  being 
studied  of  the  animal  (if  the  survey  addresses  size,  a picture  with  a measuring  tool  would  provide  evidence,  without 
having  to  remove)  before  reburying.  The  images  provide  data  in  instances  where  it  is  undesirable  to  remove 
specimens  due  to  environmental  restrictions,  possible  hazards  of  handling  an  unknown  specimen,  or  violating  state 
or  federal  law. 

Secondary 

At  this  level,  much  of  what  we  do  is  only  successful  to  the  degree  the  student  sees  how  it  is  going  to  help 
them  “in  life”.  Chances  to  make  connections  between  the  classroom  and  the  world  the  student  will  be  entering  are 
always  needed.  In  the  geometry  classroom,  the  use  of  the  shapes  and  formulas  in  construction  could  be  made 
apparent  to  the  students.  A field  trip  to  a city  will  provide  examples  of  the  role  geometry  plays  in  the  design  and 
construction  of  a city.  Images  of  different  building  and  their  functions  would  provide  the  “eye”  necessary  to  view 
the  actual  buildings.  A general  presentation  of  buildings  with  accompanying  architectural  drawings,  showing  the 
relationship  of  the  design  to  the  finished  product  provides  an  overview  of  the  trip.  For  example,  showing  an  image 
of  a finished  hotel  / motel  constructed  of  individual  modules  which  were  built  off  site  and  imported  into  the 
construction  site  as  prefabricated  units  could  be  used  to  illustrate  how  geometry  is  necessary  to  determine  the 
relationships  and  measurements  of  each  unit.  Students  on  the  trip  could  then  look  for  current  examples  of  this  type 
of  construction.  Students  given  the  task  of  looking  for  buildings  showing  examples  of  the  architect’s  use  of  golden 
rectangles,  would  be  shown  an  example  from  an  image  of  a building  with  this  feature  and  would  then  search  the  city 
for  further  examples.  Other  mathematical  principles  that  can  be  illustrated  include  the  parallel  postulate,  instances 
of  geometric  solids,  and  use  of  similar  triangles  to  determine  height.  When  on  the  trip,  documenting  geometric 
application  is  done  by  taking  pictures  of  a variety  of  buildings  to  provide  a wide  range  of  shapes  used.  Images 
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would  replace  sketches  and  written  notes.  These  images  would  then  be  imported  into  a draw  program  for  future 
instruction.  Pictures  imported  into  a draw  program  can  be  analyzed  to  complete  the  given  task.  This  could  be  done 
by  manipulating  and  studying  the  characteristics  of  the  shapes,  using  a grid  superimposed  on  an  image  to  develop 
formulas  for  geometric  aspects  of  the  building,  or  to  show  elements  of  projective  geometry  by  drawing  vanishing 
points.  A database  on  buildings  could  be  developed  and  might  include  the  function  of  the  building,  the  dimensions 
(including  square  footage  of  floor  space  & cubic  volume  of  usable  space)  image  of  buildings  and  image  of  the 
building’s  environment  to  show  esthetic  qualities. 

Teachers:  Using  Digital  Images  After  the  Field  Trip 

The  images  also  have  uses  that  are  not  included  in  the  planning  of  the  trip.  The  images  could  be  used  to 
enhance  future  instruction,  evaluate  the  objectives  related  to  the  field  trip,  and  provide  documentation  of  the  quality 
of  instruction  provided  by  the  trip. 

Teachers  can  build  a collection  of  the  digital  images  taken  on  the  field  trip  for  use  in  follow-up  activities 
either  identified  in  the  planning  stage,  when  questions  that  were  not  planned  occur  or  at  a later  date  if  questions  that 
relate  back  to  the  experience  arise.  Images  from  several  field  trips  could  be  combined  to  compile  a resource  file  for 
use  with  future  classes.  On  field  trips  often  times  unique  things  occur  that  the  teacher  may  want  to  preserve  for  use 
in  future  classroom  presentations.  A bee  keeper  may  be  requeening  or  medicating  a hive;  a bird  or  shell  might  be  on 
the  beach  that  is  unusual  or  not  in  season  during  the  field  trip;  or  a building  under  construction  would  offer  an 
opportunity  for  taking  pictures  to  compare  later  with  the  finished  product. 

Images  from  the  trip  can  be  used  when  developing  evaluation  instruments  of  the  content  delineated  by  the 
curricular  goals  and  objectives  for  which  the  field  trip  provided  instruction.  At  the  primary  level  a child  could 
demonstrate  their  oral  skills  by  giving  show  and  tell  reports  of  pictures  taken  on  the  field  trip.  At  the  middle  grades 
and  secondary  level  test  questions  could  present  a visual  image  and  ask  the  students  to  identify  or  discuss  the 
situation  (e.g.,  include  the  identification  and  the  use  of  plants  for  medical  use,  both  historically  and  presently). 
Demonstrations  of  a student’s  ability  to  use  research  tools  could  be  documented  by  presenting  the  student  with  a 
picture  of  a plant  or  animal  and  have  him/her  use  the  appropriate  field  guide  to  identify  the  specimen  and  its 
characteristics.  For  instance,  birds  in  flight  are  often  shown  in  silhouette  in  the  field  guide.  This  is  very  hard  for  the 
neophyte  to  see  and  remember  in  the  field.  Looking  at  a picture  they  can  begin  to  compare  what  they  saw  with  what 
is  shown.  Teachers  can  have  students  include  images  in  papers  and  reports  of  investigations  that  will  be  used  in 
portfolios  and  other  authentic  assessment  situations  of  the  student.  Images  collected  from  all  trips  could  also 
provide  excellent  documentation  of  the  quality  of  instruction  the  teacher  is  providing  and  could  be  included  in  a 
portfolio  of  their  own  work  when  needed. 

The  teacher  could  construct  a virtual  reality  (VR)  field  trip  to  be  used  as  part  of  the  advanced  organizer  to 
prepare  students,  to  be  used  after  the  trip  to  debrief  the  student,  and  discussion  what  was  and  was  not  seen.  A VR 
field  trip  could  provide  students  who  missed  the  trip  an  opportunity  to  participate  in  the  planned  experience.  In 
addition  it  could  be  used  to  allow  the  parent  to  see  what  the  students  will  experience. 

As  teachers  use  and  become  familiar  with  the  camera  they  will  find  that  using  the  digital  camera  frees  them 
from  the  worries  and  expense  of  film  based  pictures,  the  time  consumption  of  processing  images,  and  expands  their 
ability  to  manipulate  the  pictures  and  insert  them  into  classroom  presentations. 
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Use  Technology  to  Facilitate  Work  Ethic  Instruction 

Hong  Feng 
Roger  B.  Hill 
The  University  of  Georgia 

John  Dewey  claims  that  the  goal  of  education  is  intellectual  and  moral  development.  Work  ethic  has 
become  increasingly  important  as  a result  of  technological  development  and  globalization,  which  change  the  nature 
of  American  work  and  workplace.  Greater  mobility,  less  direct  supervision,  and  high-discretion  work  environments 
have  made  it  necessary  for  people  have  strong  work  attitude.  Researches  also  indicate  that  people’s  attitude  greatly 
influences  if  organizations  is  competitive  in  the  global  market.  (Ali  & Azim  1993).  In  the  Workplace  Know-how 
identified  by  the  SCANS  (The  Secretary's  Commission  on  Achieving  Necessary  Skills)  Report  2000  for  American, 
interpersonal  skills  is  listed  as  one  of  the  important  competencies  an  effective  worker  should  have.  In  addition, 
personal  qualities  such  as  individual  responsibility,  self-esteem,  sociability,  self-management,  and  integrity,  is  listed 
as  one  of  the  three  fundamental  skills  and  qualities.  Judy  and  Amico  (1997)  believe  “behave  dependably”  is  one  of 
the  key  factors  for  basic  education  t o ensue  the  long-term  success  of  workforce.  As  a result,  work  ethic  has  become 
an  important  issue. 

Work  ethic  is  often  defined  as  “a  cultural  norm  that  advocates  being  personally  responsible  for  the  work 
that  one  does  and  is  based  on  the  belief  that  work  has  intrinsic  value.”  (Cherrington,  1980;  Colson  & Eckerd,  1991; 
Quinn,  1983;  Yankelovich  & Immerwahr,  1984)  It  is  often  related  to  character  development  and  employability 
skills.  The  key  components  of  work  ethic  are  initiative,  interpersonal  skills,  and  dependability  (Hill,  2004).  In 
modern  workplaces,  employees  need  to  take  initiative,  to  know  how  to  cooperate  with  others  and  to  be  dependable. 
Despite  the  importance  of  work  ethic  in  work,  few  schools  in  America  have  systematic  work  ethic  instructions. 

Many  teachers  feel  it  hard  to  teach  ethics.  Research  is  needed  to  identify  the  content  and  method  of  work  ethic 
instruction.  (Hill  2003,  2004).  Dr.  Roger  Hill  attempted  to  put  his  research  in  work  ethic  into  practice  to  facilitate 
work  ethic  instruction.  With  the  wide  spread  of  Internet  technology,  many  instructors  have  taken  the  great  advantage 
of  the  convenience  it  bring  about  to  facilitate  teaching  and  learning.  Dr.  Hill  has  been  trying  to  utilize  technology  to 
facilitate  work  ethic  instruction.  Some  examples  of  his  effort  are:  collaboration  with  John  Marshall  High  School,  the 
creation  of  a work  ethic  instruction  site,  and  opening  of  a graduate  course  on  work  ethic. 

While  attending  a conference,  Dr.  Hill  met  Lynn  Mrotek,  who  had  developed  a Career  Pathways  Program 
with  her  colleagues  in  John  Marshall  High  School  in  Milwaukee,  Wisconsin.  Work  ethic  was  an  important 
component  in  their  Career  Pathways  Program,  which  used  Comprehensive  9- 12th  grade  career  discovery 
curriculum.  It’s  a funded  project  by  Carl  Perkins  Grant  and  supplemented  by  district  STW  funds.  All  students  in  the 
high  school  participated  in  the  program.  It  used  project-based  assessments  enhanced  by  technology.  It  also 
integrated  academic  and  vocational  skills. 

To  meet  the  school’s  special  needs,  Dr.  Hill  developed  a 10-day  unit,  which  includes  case  studies,  class 
discussion,  and  learning  activities.  This  part  was  in  printed  form  at  the  time  it  was  developed.  With  high  demand 
for  the  material,  it  has  been  transferred  into  electronic  version  and  put  online  with  a password  protection. 

To  supplement  the  unit  material,  Dr.  Hill  created  Work  Ethic  site  (URL:  http://www.coe.uga.edu/workethic). 
Work  Ethic  site  offers  comprehensive  information  on  work  ethic.  The  site  includes  online  lessons,  work  ethic 
history,  Occupational  Work  Ethic  Inventory  (OWEI)  and  Employability  Skill  Assessment  (ESA).  Both  OWEI 
and  ESA  are  dada-base  driven  and  self- scoring  instruments.  The  10-day  unit  originally  developed  for  John 
Marshall  High  School  is  also  on  the  site.  In  addition,  there  are  also  references  for  work  ethic  research  and  links 
to  Character  Counts  and  Thought,  Word,  and  Deed.  The  Chinese  version  of  the  site  is  also  under  construction. 

The  pilot  test  on  the  10-day  unit  conducted  in  John  Marshall  High  School  indicates  that  76%  students  rated  it 
helpful  and  24%  rated  it  not  helpful.  Some  feedback  related  to  work  ethic  site  are:  organization  of  the  web  site  and 
self  pacing  of  internet  format  were  good;  but  there  were  some  problems  with  using  the  World  Wide  Web,  computer 
lab,  computer  operation,  and  getting  to  web  materials;  the  site  needed  more  graphics  and  examples  and  used  colored 
text  to  emphasize  key  ideas.  Some  data  based  on  email  messages  received  has  also  been  gathered  with  regard 

to  the  site.  Data  analysis  indicates  that  the  work  ethic  site  has  produced  wide-range  and  very  positive  effects.  People 
from  many  parts  of  the  world  are  able  to  get  access  to  the  site  and  get  direct  benefit  in  their  work  or  study.  99% 
people  commented  that  the  site  was  very  valuable  and  useful.  Among  the  population,  21.5%  people  said  they  like 
the  whole  site  very  much  and  have  used  information  of  the  site  for  teaching,  learning,  training  and  research 
purposes;  42.4%  people  were  especially  interested  in  using  the  10-day  unit  for  instruction  and  training;  17.9% 
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people  required  detailed  information  on  interpretation  of  the  scores  resulting  from  taking  the  two  online  tests;  15.2% 
people  sake  further  advices  or  information  on  the  topic  from  Dr.  Hill.  Another  effort  Dr.  Hill  made  in  technology 
integration  of  work  ethic  instruction  is  the  opening  of  a graduate  course  “Workforce  Ethics  for  a Technological 
World”.  This  is  a 3 credits  course.  Students  meet  the  instructor  face-to-face  only  at  the  beginning,  in  the 
middle,  and  at  the  end  of  semester.  The  rest  of  sessions  are  conducted  through  Horizon  Wimba  through  which 
students  “meet”  the  instructor  each  week.  During  each  class  session,  students  can  listen  to  instructor’s 
presentation  while  watching  PowerPoint  slides  and  some  other  materials.  Student  can  also  do  individual  or 
group  presentation.  In  addition,  students  and  instructor  can  communicate  with  each  other  either  through 
microphone  or  chat.  There  is  also  WebCT  discussion  board  open  to  the  class.  All  classes  are  archived.  Email 
and  Instant  Messenger  are  also  available.  Students  submit  homework  through  email.  As  a result  of  the  powerful 
technology,  students  in  this  class,  most  of  whom  are  working  people  and  live  far  away,  can  take  the  course 
while  working  full  time.  All  students  who  have  taken  the  course  like  the  format  of  the  class  and  enjoy  taking  it. 
In  general,  Dr.  Hill’s  effort  in  integrating  technology  into  work  ethic  instruction  has  been  quite  successful. 
He  has  tried  to  take  full  advantage  of  available  technology  and  his  expertise  in  the  area  of  work  ethic  to  facilitate 
work  ethic  teaching  and  learning.  Because  of  technology,  a remote  school  can  get  instant  help  and  guidance  from  a 
university  faculty  and  people  can  get  high  quality  work  ethic  instructional  materials  anywhere  and  anytime.  With 
technology,  more  people  are  able  to  get  further  education  while  working.  Next  step  will  include  developing  a “Plan 
B”  for  those  who  cannot  reach  the  site  by  providing  printed  materials  or  through  other  ways.  It’s  also  necessary  to 
add  a link  to  score  interpretation  for  OWEI  and  ESA  tests,  which  will  provide  more  convenience  for  test  takers. 
Meanwhile,  further  development  of  the  site  including  instructional  units  is  also  crucial. 
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Abstract 

Usability  testing  of  a Web  site  is  commonly  conducted  as  a formative  evaluation  methodology  in  order  to 
identify  significant  usability  problems  that  need  to  be  fixed.  Usability  testing  can  also  be  used  to  summatively 
evaluate  the  effectiveness  of  a Web  site , i.e.,  to  determine  if  it  is  working  well  for  the  target  audience.  Some  debate 
exists  as  to  how  many  tasks  are  needed  to  make  a decision  regarding  the  effectiveness  of  a Web  site , and  the 
question  is  raised:  Can  the  number  of  tasks  be  adapted  based  on  user  performance  during  a usability  test?  This  is 
particularly  significant  for  Web  sites  which  are  very  large  and  complex,  and  which  would  require  a large  number  of 
usability  tasks  to  evaluate  them  thoroughly.  A Bayesian  decision  model,  the  Sequential  Probability  Ratio  Test 
(SPRT),  was  investigated  as  an  adaptive  method  for  determining  how  many  tasks  are  needed  during  usability 
evaluation  of  a complex  informational  Web  site. 

Twenty-five  undergraduate  students  in  the  Indiana  University  School  of  Education  were  tested  to  determine 
if  a part  of  the  Web  site  was  working  well  for  teacher  education  students.  Each  user  was  tested  with  20  randomly 
selected  tasks  during  a one  hour  session.  The  SPRT  was  applied  retrospectively  to  decide  when  testing  of  each 
subject  could  have  been  stopped  to  reach  a decision  of  whether  the  site  is  working  well  for  that  subject.  Results 
show  that  the  decisions  reached  using  SPRT  are  highly  consistent  with  the  decisions  reached  by  testing  the  users 
with  all  20  tasks.  When  the  SPRT  was  applied,  users  could  have  stopped  a usability  test  with  an  average  of  12  tasks. 
Moreover,  a decision  could  be  reached  about  Web  site  effectiveness  with  5 randomly  sampled  users  in  our  study.  In 
a summative  evaluation  of  a complex  Web  sites  effectiveness,  the  SPRT  increases  the  efficiency  of  the  usability 
testing  by  utilizing  only  as  many  tasks  and  users  as  necessary  to  reach  a confident  decision.  However,  reducing  the 
numbers  of  users  and  tasks  may  be  counterproductive  if  one  is  interested  in  identifying  usability  problems.  In  that 
case,  more  users  and  tasks  are  likely  needed  during  formative  evaluation. 

Introduction 

A usability  test  of  a Web  site  is  commonly  conducted  to  either  detect  problems  requiring  improvement  or  to 
determine  the  effectiveness  of  the  Web  site,  i.e.,  if  it  is  working  well  for  the  target  audience.  Problem  detection  is 
more  likely  to  occur  as  a formative  evaluation  process  during  the  development  of  a Web  site  since  developers  intend 
to  correct  problems  identified  through  usability  testing.  Some  researchers  recommend  that  testing  five  users  is 
enough  to  detect  most  of  the  problems  of  a Web  site  (Nielsen  & Landauer,  1993);  however,  this  recommendation 
has  been  criticized  by  other  researchers  (Spool  & Schroeder,  2001;  Woolrych  & Cockton,  2001),  who  claim  that  the 
lack  of  homogeneity  of  users  and  tasks  often  require  many  more  users  in  order  to  identify  major  problems. 

On  the  other  hand,  a Bayesian  decision  model,  the  Sequential  Probability  Ratio  Test  (SPRT)  that  was 
originally  developed  by  Wald  (1947),  has  been  employed  successfully  in  usability  tests  to  determine  the 
effectiveness  of  a Web  site  as  a summative  evaluation  process  (Frick  et  al.,  2003).  Wald’s  SPRT  methods  are  well- 
known  in  statistics  and  in  manufacturing  quality  control.  The  SPRT  provides  rules  to  stop  testing  and  choose  one  of 
two  discrete  alternatives  using  a priori  decision  error  rates.  The  purpose  of  this  study  is  to  determine  if  SPRT  can  be 
used  as  a valid  tool  to  improve  the  efficiency  of  Web  site  usability  testing  by  testing  as  few  tasks  as  necessary  to 
make  a conclusion  on  the  effectiveness  of  a complex  informational  Web  site  comprised  of  about  6,000  Web  pages. 

Literature  Review 

Usability  has  many  similar  definitions  but  one  that  has  broad  recognition  with  the  U.S.  Government  is  the 
definition  provided  by  the  US  Department  of  Health  and  Human  Services  (HHS,  2004):  “Usability  is  the  measure  of 
the  quality  of  a user's  experience  when  interacting  with  a product  or  system  — whether  a Web  site,  a software 
application,  mobile  technology,  or  any  user-operated  device.”  (n.p.) 
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There  are  many  other  definitions  and  descriptions  that  describe  the  multiple  attributes  of  the  user 
experience  in  the  literature,  but  the  three  most  common  as  cited  by  the  International  Standards  Organization  (ISO 
9241-11,1998)  are: 

1.  Effectiveness  - accuracy  and  completeness  of  achieving  the  goals 

2.  Efficiency  - speed  and  resources  expended  in  achieving  the  goals 

3.  Satisfaction  - Does  the  target  audience  like  using  the  system? 

Since  the  1980’s  there  has  been  a profound  shift  in  attention  to  user  needs,  as  noted  by  the  emergence  of 
usability  testing  and  usability  test  facilities.  Usability  testing  has  improved  project  quality,  accelerated  project 
delivery  and  provided  dramatic  cost  savings,  capturing  the  interest  of  both  designers  and  managers  (Nielsen,  1993; 
Shneiderman  & Plaisant,  2004).  During  this  time  many  usability-laboratory  advocates  split  off  from  their  academic 
roots  into  their  own  small  businesses  and  consulting  practices  or  joined  larger  firms  that  had  usability  labs.  This 
trend  has  created  a consulting  community  that  provides  testing  for  hire  and  currently  remains  an  active  service. 

With  that  growth  and  maturity  has  come  a shift  in  usability  procedures.  Dumas  & Redish  (1999)  point  out  in  the 
preface  to  their  revised  edition  of  the  shift  to  more  informal  testing  of  usability:  little  or  no  videotaping,  not  logging 
every  action,  and  smaller  groups  for  each  test  session.  One  reason  they  claim  is  that  “more  acceptance  of  the  fact 
that  the  value  of  usability  testing  is  in  diagnosing  problems  rather  than  validating  products”  (p.  xi)  reinforcing  that 
large  numbers  of  test  participants  are  not  necessary  to  feel  confident  that  enough  problems  have  been  identified.  Yet 
there  is  still  a need  for  summative  evaluation  of  products  to  make  decisions  on  when  to  release  the  product  to  users 
and  the  general  public. 

Due  to  the  costs  associated  with  usability  testing,  the  practitioner  community  has  focused  on  the  efficiency 
of  usability  testing.  Perhaps  one  of  the  first  widely  recognized  approaches  to  improving  usability  testing  efficiency 
was  Jakob  Nielsen’s  “Discount  Usability  Engineering”  (Nielsen  1993),  which  advocated  what  he  called  “the  good” 
usability  methods  which  do  not  necessarily  give  perfect  results  versus  the  “best”  which  may  result  in  no  usability 
testing  being  performed  at  all.  This  method  is  based  on  four  techniques: 

• User  and  task  observation 

• Scenarios 

• Simplified  thinking  aloud 

• Heuristic  evaluation 

As  noted  by  Shneiderman  and  Plaisant  (2004),  one  of  the  more  controversial  aspects  to  the 
recommendation  is  use  of  only  three  to  six  test  participants  in  a round  of  usability  testing. 

How  Many  Participants  are  Needed? 

Much  has  been  made  in  the  Human  Factors  / Usability  community  in  the  last  decade  about  the  proper 
number  of  users  needed  to  perform  a usability  test  on  a Web  site.  Much  of  the  research  in  the  1990’s  suggested  that 
five  participants  will  yield  80-85  percent  of  the  findings  in  a usability  test  (Nielsen  1992,  1994,  2000;  Virzi  1990, 
1992;  Lewis  1994).  These  recommendations  are  based  on  the  use  of  a Poisson  binomial  probability  distribution 
(Nielsen  & Landauer,  1993). 

Two  of  the  more  popular  usability  testing  guidebooks,  the  Handbook  of  Usability  Testing  (Rubin  1994)  and 
A Practical  Guide  to  Usability  Testing  (Dumas  & Redish  1999)  discuss  how  many  users  are  needed  in  usability 
testing.  Rubin  noted  in  regard  to  sample  sizes  “to  achieve  generalizable  results  for  a given  target  population. . . one 
may  need  to  test  10-12  participants  per  condition  to  be  on  the  safe  side,  a factor  that  might  require  one  to  test  40  or 
more  participants  to  ensure  statistically  significant  results.”  (p.  29)  Rubin  also  noted  that  it  is  often  inappropriate  or 
impossible  to  use  classical  experimental  design  procedures  to  conduct  usability  tests  in  the  fast  paced  development 
environment.  Rubin  supports  the  Virzi  and  Nielsen  testing  recommendations,  but  takes  it  one  step  further  - 
recommending  four  participants  per  treatment  group. 

Dumas  and  Redish  (1999),  one  of  the  more  popular  usability  guides  whose  first  edition  was  published  in 
1993,  discussed  how  many  people  to  include  in  usability  testing  as  part  of  the  whole  usability  testing  process. 
Starting  from  a task  analysis  perspective,  Dumas  and  Redish  first  suggested  identifying  user  profiles  and  then 
selecting  subgroups  that  will  need  to  be  tested.  Characteristics  that  are  important  for  those  subgroups  are  then 
defined  and  the  most  critical  characteristics  in  participants  are  selected  for  testing.  These  are  used  to  determine  the 
range  of  participants,  which  then  has  an  impact  upon  how  many  participants  need  to  be  selected  for  usability  testing. 

When  recommending  an  actual  number  of  participants  to  use,  Dumas  and  Redish  (1999)  cite  the  Nielsen  & 
Molich  (1990)  along  with  the  Virzi  study  (1992)  that  claim  3 to  5 users  are  enough.  Dumas  and  Redish  seem  to 
hedge  a bit  by  noting  most  of  the  major  problems  are  found  with  3 to  5 users  but  most  usability  tests  are  with  6 to  12 
participants.  Their  recommendation  of  “3  participants  for  each  subgroup  is  probably  an  absolute  minimum”  (p.  128) 
seems  to  indicate  they  feel  the  3 to  5 recommendation  is  best  applied  to  the  sub-group  level,  although  this  is  as  close 
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as  they  come  to  a specific  recommendation  pointing  out  that  you  have  to  balance  time,  money  and  information 
gained  when  performing  usability  tests. 

More  recently  there  have  been  challenges  to  the  assumptions  about  4 to  5 users  being  enough,  (Caulton 
2001;  Woolrych  and  Cockton  2001;  Spool  2001;  Molich  et  al  2004).  These  studies  consider  the  effect  of  sub-groups 
upon  the  homogeneity  assumption,  the  severity  of  the  problem  and  the  frequency  at  which  it  might  occur,  and  the 
effect  of  different  evaluators.  The  latter  two  studies  also  point  out  potential  issues  that  should  be  considered  for 
large,  commercial  web  sites  including  visiting  different  portions  of  large  sites  and  potentially  conducting  different 
tasks.  In  larger  Web  sites  the  validity  of  the  tasks  chosen  to  test  compared  to  the  full  site  usability  could  be  a greater 
variable  than  having  the  correct  number  of  users. 

The  SPRT  was  developed  by  Abraham  Wald  and  utilized  by  the  U.S.  government  in  World  War  II  for 
quality  control  of  weapons.  Wald  demonstrated  that  by  making  observations  sequentially  and  applying  decision 
rules  after  each  observation,  then  about  half  as  many  observations  on  the  average  are  required  to  make  a decision 
versus  conventional  testing  with  predetermined  fixed  sample  sizes.  SPRT  does  not  take  into  account  item  difficulty 
variability  or  chances  of  guessing  like  more  complex  models  such  as  item  response  theory  (IRT).  However,  SPRT 
does  not  require  the  large  amount  of  testing  in  advance  to  determine  item  response  functions  that  IRT  models 
require  (Frick,  1992). 

Application  of  SPRT  to  Usability 

Limited  literature  exists  that  addresses  the  number  of  users  needed  to  conclude  if  a Web  site  is  working 
well  (also  noted  in  Frick  et  al.  2003).  Perhaps  because  the  emphasis  has  been  upon  problem  identification  during 
development  (more  formative  evaluation),  there  seems  to  be  little  research  on  providing  evidence  that  a Web  site  is 
effective  in  allowing  users  to  reach  their  goals  (summative  evaluation). 

Frick  may  have  been  the  first  to  apply  SPRT  to  usability  testing.  A 2001  study  (Frick  et  al.)  looked  into 
applying  Wald’s  SPRT  to  Web  site  usability  testing.  By  focusing  on  Web  site  effectiveness  (more  of  a summative 
evaluation)  instead  of  problem  identification,  the  authors  applied  SPRT  to  see  if  the  average  sample  size  to  make  a 
decision  on  when  to  stop  testing  would  be  reduced. 

This  study  took  31  subjects  and  asked  them  to  perform  20  random  tasks  selected  from  a pool  of  over  330 
tasks  associated  with  the  Indiana  University  Bloomington  (IUB)  Web  site.  Subjects  were  recruited  on  a proportional 
basis  according  to  the  population  served  by  the  IUB  Web  site.  The  SPRT  parameters  were  set  a priori  at:  success 
level=.90,  failure  level=.50,  a error=.05,  error=.05  and  then  applied  at  the  task  level  for  all  31  participants  post 
hoc. 

Application  of  post  hoc  SPRT  would  have  resulted  in  30  of  the  31  decisions  to  be  the  same  as  though  all  20 
tasks  had  been  put  into  the  SPRT  formula.  Since  the  expected  agreement  was  90  percent  (l-(a+/?)),  the  SPRT  made 
fewer  classification  errors  (3.2  percent)  than  the  expected  error  rate  of  10  percent. 

Using  the  same  SPRT  parameters  and  applying  at  the  user  level,  the  decision  on  Web  site  effectiveness 
could  have  been  made  after  only  four  subjects.  The  study  concluded  that  the  IUB  Web  site  was  not  effective  with  the 
SPRT  parameters  used  when  using  the  “short”  SPRT  test  (4  users)  or  when  it  was  applied  to  all  31  users.  The  study 
also  calculated  that  applying  SPRT  would  only  take  12-20  percent  of  the  testing  time  versus  what  was  defined  as  a 
typical  usability  test  (10  subjects  with  15  tasks).  This  number  should  be  viewed  cautiously  however,  for  if  the 
Discount  Usability  Engineering  procedures  suggested  by  Nielsen  are  used  (five  users)  the  advantage  would  only  be 
slightly  less. 

A second  study  by  Frick  et  al.  (2004)  looked  at  specifically  applying  SPRT  to  the  number  of  users  on  the 
IU  Library  Web  site  to  determine  the  effectiveness  of  its  electronic  card  catalog  system  (IUCAT).  A total  of  51 
people  were  selected  through  a stratified  convenience  sample  and  were  asked  to  attempt  two  specific  but 
representative  tasks  to  find  holdings  listed  in  the  catalog  using  the  search  function  for  IUCAT.  The  two  tasks  chosen 
took  about  10-15  minutes  total,  a design  chosen  so  that  more  user  tests  could  be  obtained  for  the  analysis.  Participant 
test  results  were  randomized  to  avoid  possible  bias  from  the  results.  The  results  were  analyzed  retroactively  to 
determine  how  many  participants  would  be  needed  to  make  a decision  on  effectiveness.  Then  various  SPRT 
parameters  were  changed  to  see  the  effect  upon  the  number  of  subjects  needed  to  reach  a conclusion.  Utilizing 
baseline  SPRT  factors  of  success  level=.90,  failure  level=.60,  a error=.05  and  error=.05  a decision  that  the  search 
engine  was  effective  could  have  been  made  after  12  subjects.  This  was  the  same  conclusion  that  would  have  been 
reached  within  the  results  from  all  5 1 participants  had  been  used. 

Next,  the  study  looked  into  reducing  the  zone  of  indifference  (success  minus  the  failure  level).  As  the  zone 
of  indifference  was  reduced,  more  subjects  were  needed  to  make  a decision.  Likewise  when  the  alpha  and  beta  error 
rates  were  reduced,  more  subjects  were  needed  to  make  a decision  in  the  Frick  et  al.  (2004)  study. 
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Research  Questions 

This  study  intended  to  answer  the  following  research  question:  When  conducting  a usability  test  with 
random  task  selection,  how  does  the  SPRT-based  method  of  decision-making  compare  to  the  results  of  conventional 
usability  testing?  In  other  words,  if  tasks  are  randomly  selected  from  a large  pool  for  usability  testing,  does  the 
SPRT  method  of  deciding  when  to  stop  and  make  a decision  agree  with  the  decision  that  would  have  been  reached 
with  all  tasks?  A secondary  question  was:  Can  the  number  of  users  also  be  adapted  by  use  of  the  SPRT? 

Significance  of  Study 

Wald’s  SPRT  was  classified  as  a defense  secret  by  the  U.S.  government  during  World  War  II  due  to  its 
practical  effectiveness  in  making  product  quality  control  decisions.  By  applying  SPRT  in  usability  tests,  researchers 
do  not  have  to  predetermine  how  many  tasks  to  test  before  the  usability  testing  starts.  Instead,  decisions  of  when  to 
stop  testing  will  be  reached  after  analyzing  data  accumulated  from  the  usability  testing.  Often  SPRT  leads  to  quick, 
but  highly  accurate,  decisions  regarding  the  effectiveness  of  a product  with  fewer  tasks  to  test.  Therefore  it  may 
increase  the  efficiency  of  Web  site  usability  testing. 

Usability  testing  can  be  very  costly  in  both  money  and  time.  If  the  SPRT  proves  to  be  a reliable  method  in 
usability  testing  for  determination  of  a Web  site’s  effectiveness,  it  may  improve  the  efficiency  of  usability  testing 
and  lower  related  costs  substantially. 


Methods  of  Usability  Testing 

Usability  testing  methodology  was  used  to  evaluate  the  School  of  Education  Web  site  (Dumas,  1999).  This 
approach  involved  having  authentic  users  perform  authentic  tasks  using  the  system,  while  a facilitator  guided  the 
usability  session.  An  evaluator  recorded  the  users’  actions  and  comments  while  tasks  were  completed.  During  the 
testing,  users  were  asked  to  perform  a think-aloud  protocol  to  help  the  evaluators  understand  their  behaviors  and 
gain  insight  into  the  design  of  the  Web  site  (Ericsson,  1993). 

Sessions  were  performed  on  an  individual  basis  with  each  session  lasting  approximately  one  hour. 
Following  the  last  session,  qualitative  and  quantitative  data  were  analyzed  and  summarized,  and  problem  areas  were 
identified.  Participants  in  this  study  were  provided  with  five  extra  credit  points  from  their  class  instructors. 

Participants 

Participants  for  the  study  were  recruited  by  the  researchers  from  School  of  Education  classes.  One  of  the 
target  audiences  is  current  Teacher  Education  students  who  are  at  the  undergraduate  level.  The  following 
demographic  criteria  were  used  as  guidelines: 

• Gender  - approximately  75%  female,  25%  male 

• Student  status  - undergraduate  level  students 

• School  of  Education,  Teacher  Education  majors 

A sample  of  25  participants  was  obtained  from  36  volunteers  from  four  undergraduate  classes 

Procedures 

Participants  were  asked  to  read  and  sign  Indiana  University  Human  Subjects  consent  form  which  included  a 
brief  description  of  the  usability  session,  the  user’s  risks,  benefits,  confidentiality  and  the  researchers’  contact 
information.  Users  were  then  asked  to  fill  out  a brief  survey  which  included  demographic  questions,  such  as 
university  class  standing  and  prior  experiences  with  the  School  of  Education  Web  site. 

Tasks  were  completed  one  at  a time  and  recorded  as:  1)  success,  if  the  answer  to  the  question  was  found  on 
the  Web  site,  or  2)  failure  due  to  expiration  of  3-minute  time  limit,  or  3)  failure  because  the  user  gave  up,  or  4) 
failure,  if  the  answer  was  not  found  even  though  the  user  thought  she  or  he  had  done  so.  Following  the  session,  the 
users  were  asked  a series  of  post-session  questions  which  helped  capture  their  overall  experience  with  the  Web  site: 
1)  Now  that  you  have  used  the  School  of  Education  Web  site,  what  are  your  overall  impressions  of  it?  2)  If  you 
could  pick  one  aspect  of  the  site  that  you  did  not  like,  what  would  it  be?  3)  What  is  one  feature  about  the  site  that 
you  found  positive  and  liked?  The  users’  qualitative  post-session  answers  helped  the  research  team  to  better 
understand  specific  concerns  users  may  have  with  the  system  in  areas  that  were  not  tested  in  the  session. 

Sessions  were  performed  using  Internet  Explorer  6.0  and  the  Windows  XP  operating  system  on  an  Intel 
Pentium  IV  computer  in  the  School  of  Education  1ST  usability  lab. 

Tasks 

Each  user  was  asked  to  complete  20  tasks  that  were  randomly  drawn  from  a pool  of  120  tasks  generated 
from  a prior  needs  analysis  study.  The  tasks  selected  were  ones  that  were  considered  relevant  to  teacher  education 
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students,  as  compared  with  those  for  alumni,  faculty  or  K-12  professionals.  Since  tasks  were  randomly  selected, 
there  were  very  few  tasks  that  were  the  same  for  most  users.  Tasks  also  were  created  in  order  to  test  users  interacting 
with  specific  features  of  the  system.  Users  were  asked  to  complete  as  many  of  the  20  tasks  as  possible  within  a one- 
hour  user  session.  Each  user  was  given  three  minutes  to  complete  a task.  This  time  limitation  allowed  a user  enough 
time  to  attempt  all  20  tasks  within  an  hour. 


Methods  of  SPRT 

The  task  data  from  each  subject  were  run  through  the  Web  Tool  for  Sequential  Bayesian  Decision  Making 
(Frick,  2003).  The  settings  for  the  Bayesian  Decision  Making  tool  were  as  follows: 

• Alternative  A:  Web  site  is  working  well 

• Alternative  B:  Web  site  is  NOT  working  well 

• Minimum  proportion  of  success  if  Alternative  A is  true:  .85 

• Maximum  proportion  of  success  if  Alternative  B is  true:  .60 

• Error  rate  for  choosing  Alternative  B when  A is  really  true  = Alpha  error  = .05 

• Error  rate  for  choosing  Alternative  A when  B is  really  true  = Beta  error  = .05 

The  task  data  were  used  in  two  ways.  First,  the  total  number  of  successes  and  the  total  number  of  failures 
per  subject  were  entered  to  determine  if  the  Web  site  worked  well,  did  not  work  well  or  if  more  tasks  were  needed  to 
make  a decision  based  on  all  of  the  tasks.  Next,  the  usability  test  was  reenacted  by  entering  the  outcome  of  each  task 
into  the  SPRT  Web  tool  one  item  at  a time  for  a given  subject.  The  SPRT  tool  would  return  a determination  after 
each  task  indicating  if  alternative  A or  B could  be  chosen,  or  if  more  tasks  were  needed  to  make  a decision.  If  more 
tasks  were  needed,  the  next  task  result  was  entered.  If  a decision  was  made,  then  the  number  of  tasks  entered  to  that 
point  was  recorded  along  with  the  breakdown  of  the  number  of  tasks  identified  as  successes  and  failures. 

Tests  were  run  comparing  the  SPRT  result  from  the  overall  (all  tasks)  test  to  the  adaptive  (task-by-task)  test 
to  evaluate  the  consistency  between  the  two  methods.  In  addition,  the  mean  success  rates  across  all  participants  for 
both  the  overall  test  and  adaptive  test  were  compared. 


Results 

Twenty-five  subjects  participated  in  the  usability  tests,  22  females  and  three  males.  The  female/male  split 
was  88  percent  and  12  percent  respectively,  which  is  approximately  1 1 percent  higher  in  females  than  the  population 
of  undergraduates  in  the  Indiana  University  Bloomington  School  of  Education  in  2003  (76.7  percent  female/23.3 
percent  male)  (Indiana  University  School  of  Education,  2004). 

The  subjects  included  11  freshmen,  11  sophomores,  two  juniors  and  one  subject  who  did  not  report  class 
rank.  The  subjects’  academic  areas  of  interest  included  Art  Education  (3),  Early  Childhood  Education  (3), 
Elementary  Education  (7),  Special  Education  (4)  and  other  (7).  When  asked  how  often  the  subjects  visited  the 
School  of  Education  Web  site,  the  following  responses  were  marked:  Very  often  (0),  Often  (2),  Seldom  (5),  Rarely 
(10),  Never  (7)  and  one  participant  did  not  answer  the  question.  This  indicates  that  most  of  the  participants  were 
unfamiliar  with  the  Web  site  used  in  the  usability  testing. 

Table  1 shows  on  agreement  of  decisions  reached  from  all  20  randomly  selected  tasks  for  each  user  and  the 
SPRT  adaptive  reenactments.  The  value  for  kappa  is  0.74  when  corrected  for  chance  agreement,  with  a significance 
level  of  p < .01  (simple  percentage  agreement  is  22/25  or  88  percent).  This  means  there  is  substantial  agreement  that 
the  same  result  is  achieved  by  using  decisions  that  were  made  using  all  20  tasks  and  the  subsets  of  those  that  were 
made  after  reenacting  the  test  using  the  adaptive  SPRT.  In  other  words,  decisions  reached  with  a subset  of  the 
randomly  selected  20  tasks  for  each  user  were  largely  the  same  as  decisions  reached  with  all  tasks.  In  16  cases 
(subjects),  the  conclusion  was  that  the  Web  site  was  not  working  well  (with  a maximum  success  rate  of  .60  or 
lower),  using  either  the  adaptive  SPRT  method  or  by  using  all  20  randomly  selected  items.  In  two  cases,  the 
conclusion  was  that  the  Web  site  was  working  well  (with  a minimum  success  rate  of  .85  or  higher),  using  either 
method.  In  four  cases,  no  decision  could  be  reached  at  the  a priori  error  rates,  after  exhausting  all  20  items,  using 
either  method.  The  three  cases  where  the  decision  outcome  differed  was  when  SPRT  concluded  the  site  was  not 
working  well,  but  the  conclusion  after  20  items  was  that  no  decision  could  be  reached.  See  Table  1. 
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Table  1.  Crosstabulation : Did  the  Web  site  work  overall  based  on  the  adaptive  SPRT?  versus  Did  the  Web  site  work 

well  with  all  tasks?  (N=25  users,  kappa  - 0.74,  p < 0.01) 

Did  the  Web  site  work  well 
with  all  tasks  ? 


No 

Yes 

No  Decision 

Total 

Did  the  Web  site  work 

No 

16 

0 

3 

19 

well  based  on  adaptive 

Yes 

0 

2 

0 

2 

SPRT? 

No  Decision 

0 

0 

4 

4 

Total 

16 

2 

7 

25 

The  researchers  observed  improvements  in  solving  tasks  in  some  participants  as  their  tests  progressed.  This 
led  the  researchers  to  consider  that  subjects  were  becoming  more  acquainted  with  the  Web  site  as  usability  tests 
proceeded.  Therefore,  the  success  rate  for  the  first  half  and  second  half  of  each  usability  test  was  determined.  Then 
these  figures  were  compared  as  seen  in  Table  2.  There  was  an  increase  from  the  first  half  success  rate  (.56)  to  the 
second  half  of  the  test  success  rate  (.65),  and  the  difference  was  about  half  a standard  deviation.  It  does  appear  that 
subjects  did  learn  where  things  were  located  on  the  Web  site  and  how  it  is  organized,  since  they  did  perform  better 
during  the  second  half  of  the  tasks.  This  may  account  for  the  three  no-decision  outcomes  when  SPRT  concluded  the 
site  was  not  working  for  those  subjects.  The  performance  of  those  three  subjects  improved  enough  that  it  was  no 
longer  possible  to  confidently  choose  between  the  two  alternatives  (Web  site  works  or  not).  For  the  majority, 
however,  the  learning  that  occurred  did  not  affect  the  overall  conclusion  of  whether  the  Web  site  was  working  or  not 
at  the  levels  specified  by  the  Web  director  (.85  vs.  .60) 


Table  2 Comparison  of  users’  task  success  rate  between  the  first  set  of  10  tasks  and  the  second  set  of  10  tasks 


N 

Minimum 

Maximum 

Mean 

Std.  Deviation 

First  random  set  of  10 

25 

0.22 

0.90 

0.56 

0.19 

Second  random  set  of  10 

25 

0.30 

1.00 

0.65 

0.19 

Table  3. 


Number  of  tasks  needed  to  stop  testing  with  the  SPRT  and  choose  an  alternative 


Participant 

Number  of 

Number 

tasks  to  stop 
testing  with 
SPRT 

#1 

6 

#2 

6 

#3 

no  decision 

#4 

13 

#5 

16 

#6 

13 

#7 

14 

#8 

20 

#9 

9 

#10 

9 

#11 

5 

#12 

17 

#13 

5 

#14 

6 

#15 

6 

#16 

6 

#17 

no  decision 

#18 

10 

#19 

13 

#20 

6 

#21 

17 

196 


#22 

10 

#23 

no  decision 

#24 

5 

#25 

no  decision 

In  Table  3 the  numbers  of  tasks  required  to  reach  an  SPRT  decision  for  each  case  are  listed.  It  can  be  seen 
that  decisions  could  be  reached  with  as  few  as  five  or  six  tasks  for  some  users,  and  in  four  other  cases,  no  decision 
could  be  confidently  reached  with  all  20  tasks.  More  tasks  would  have  been  needed  for  those  users  in  order  to  reach 
a conclusion.  Twenty  tasks  were  not  enough  in  those  cases.  Table  3 shows  how  the  SPRT  makes  it  possible  to  adapt 
the  number  of  tasks  based  on  the  performance  of  users  and  the  decision  criteria  established  in  advance. 

From  Table  3,  it  can  be  seen  that  an  average  of  about  12  tasks  were  needed  in  our  study,  but  the  range  was 
from  5 to  more  than  20  (but  we  stopped  with  those  subjects  since  the  test  session  would  have  gone  beyond  the  hour 
allocated).  If  users  are  selected  at  random  from  those  in  Table  3,  and  the  SPRT  is  applied  after  a decision  is  reached 
for  that  user,  a total  of  5 users  would  be  needed  to  reach  the  decision  that  the  Web  site  was  not  working  as  well  as 
the  Web  Director  had  hoped  (alternative  B is  chosen  for  the  Web  site).  Thus,  if  the  SPRT  had  actually  been  used  in 
this  study,  we  could  have  reached  this  conclusion  with  5 users  and  an  average  of  12  tasks  per  user  - and  we  would 
have  reached  the  same  overall  conclusion.  This  is  a significant  reduction  in  time  and  effort  in  conducting  such 
usability  tests,  when  compared  to  25  users  and  20  tasks  each. 

Discussion 

The  SPRT  requires  the  decision  maker  to  clearly  establish  the  criteria  that  will  be  used  in  making  a 
decision.  1)  What  is  the  minimal  success  rate  that  is  acceptable  to  decide  that  the  Web  site  is  working  well?  2)  What 
is  the  maximum  success  rate  that  is  acceptable  to  decide  that  the  Web  site  is  not  working  as  well?  3)  What 
probabilities  are  tolerable  for  making  erroneous  conclusions?  For  example,  had  the  success  rates  been  0.60  vs.  0.40 
in  this  study,  with  the  same  alpha  and  beta  levels  for  decision  error  rates,  then  the  conclusion  would  likely  be  that 
the  Web  site  was  working  well  - just  the  opposite  of  that  found  in  this  study.  In  fact,  success  rates  on  most  Web  sites 
are  often  less  than  50  percent,  according  to  Nielsen  (2001),  but  he  estimates  that  success  rates  have  risen  at  a rate  of 
2.5  percent  per  year  in  the  past  few  years  due  to  increased  usability  evaluation  that  appears  to  be  occurring  (Nielsen, 
2003).  Thus,  from  this  perspective,  student  performance  in  our  usability  tasks  with  this  Web  site  were  better  than 
success  rates  on  most  Web  sites.  Nonetheless,  the  success  rate  was  not  as  high  as  the  Web  Director  had  desired. 

Furthermore,  the  120  tasks  chosen  for  this  usability  test  were  those  which  were  chosen  from  a much  larger 
list  of  frequently  asked  questions  determined  from  prior  needs  analyses.  The  120  tasks  selected  were  deemed  to  be 
largely  relevant  for  the  subgroup  we  tested  (i.e.,  undergraduate  teacher  education  majors).  The  overall  Web  site  is 
designed  to  address  other  target  audiences  that  include:  prospective  students,  graduate  students,  faculty,  staff, 
associate  instructors,  alumni  of  the  School  of  Education,  and  K-12  professionals.  The  large  bulk  of  the  6,000  pages 
on  this  site  were  not  applicable  to  the  tasks  chosen.  Most  of  the  answers  to  the  120  tasks  could  be  found  on  the  Web 
sites  for  the  Office  of  Teacher  Education,  which  consist  of  approximately  560  Web  pages  or  roughly  10  percent  of 
the  overall  Web  site.  Thus,  no  inferences  should  be  made  concerning  the  effectiveness  of  the  Website  for  these  other 
audiences,  or  for  the  90  percent  of  other  Web  pages  that  are  part  of  the  overall  School  of  Education  Web  site. 

Finally,  it  should  be  noted  that  users  in  our  study  began  their  usability  session  at  the  IU  Bloomington  home 
page.  These  users  did  not  begin  at  the  School  of  Education  home  page,  and  we  did  not  tell  them  that  the  answers 
were  on  School  of  Education  Web  sites,  nor  on  the  Office  of  Teacher  Education  Web  sites.  Thus,  the  participants  in 
our  study,  who  were  largely  unfamiliar  with  School  of  Education  Web  sites,  were  in  essence  looking  for  “needles  in 
a huge  haystack”.  Indiana  University  currently  has  in  excess  of  one  million  Web  pages  in  numerous  sites,  according 
to  the  person  who  manages  the  IU  search  engine  (Percival,  2005,  personal  communication). 

Implications  for  Further  Research 

It  is  a common  goal  of  usability  testing  methodology  to  identify  problems  that  need  to  be  fixed.  While 
applying  SPRT  may  help  an  evaluator  determine  the  overall  effectiveness  of  a Web  site,  stopping  a user  session 
early  may  result  in  failure  to  identify  specific  usability  problems.  Ultimately,  the  goal  of  usability  testing  is  to 
evaluate  the  current  state  of  a system,  detect  problems,  interpret  the  reasoning  behind  the  problem,  and  diagnose  the 
problem  with  a potential  solution.  With  the  utilization  of  SPRT  in  a usability  study,  specific  problem  detection  may 
not  occur.  One  of  the  authors  has  since  explored  this  issue  in  greater  depth  in  his  dissertation  (Hebb,  2005).  In 
further  analyses  of  the  data  from  this  study,  he  found  that  with  5 users  and  an  average  of  12  tasks  that: 
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. . .this  number  of  users  and  tasks  identified  25%  of  known  usability  problems.  Likewise,  the  Poisson-based 
approach  identified  35%  of  the  problems  with  these  same  5 users  but  with  20  tasks  per  user.  For  over  10 
years,  the  prevailing  rule  of  thumb  has  been  that  4 to  5 subjects  will  identify  the  usability  problems  when 
evaluating  websites.  Results  from  this  study  indicated  that  more  subjects  may  be  needed  for  identifying  80- 
85%  of  website  usability  problems.  Results  also  indicated  that  the  SPRT  is  more  cost-effective  than  the 
Poisson  model  if  summative  evaluation  and  not  problem  identification  is  the  goal  of  usability  testing,  (p.  vi) 

These  findings  suggest  that  efficiency  in  terms  of  minimizing  numbers  of  users  or  tasks  in  usability  testing 
is  counterproductive  if  the  goal  is  to  identify  usability  problems.  In  the  Hebb  (2005)  study,  he  identified  83  unique 
problems  with  all  25  users  and  20  randomly  selected  tasks  per  user.  If  10  users  are  chosen  at  random,  52  unique 
problems  are  identified  (63  percent),  and  if  5 users  are  chosen  only  29  unique  problems  were  uncovered  (35 
percent). 

It  appears  from  Hebb’s  data  that  if  problem  identification  is  the  goal  of  usability  testing,  then  the  five-user 
rule  that  Nielsen  and  Landauer  have  suggested  may  badly  underestimate  the  number  of  subjects  needed  to  identify 
80  percent  of  usability  problems.  More  research  is  clearly  needed  here. 

It  does  appear  from  the  Frick  et  al.  (2003)  study  that  the  SPRT  can  be  effectively  used  when  a Website  has 
few  usability  problems.  In  that  study,  the  search  engine  for  the  electronic  card  catalog  was  evaluated.  This  search 
engine  has  evolved,  been  improved,  and  has  been  in  use  for  over  15  years  (even  before  the  Web  came  about).  In 
other  words,  if  most  usability  problems  have  been  fixed  in  a product,  then  the  SPRT  can  be  an  efficient  method  of 
verifying  product  effectiveness  in  summative  evaluation.  For  formative  evaluation,  however,  fewer  numbers  of 
subjects  and  tasks  appear  to  be  detrimental  to  problem  identification. 
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The  Need  for  an  Educational  Systems  Theory 

As  the  ‘No  Child  Left  Behind’  (NCLB)  legislation  is  being  implemented,  K-12  schools  in  America  face 
increasing  pressure  to  meet  state  standards.  Successful  change  is  imperative  for  schools  classified  as  “failing,”  since 
the  consequences  of  repeated  failure  can  result  in  school  closure.  How,  then,  should  changes  in  school  systems  be 
planned  to  obtain  better  results?  What  principles  could  be  used  to  predict  possible  consequences  of  change 
strategies? 

Changes  in  educational  policies  have  the  potential  to  impact  a large  number  of  students  within  an 
educational  system  across  extended  time  periods.  The  stakes  for  such  changes  are  high,  since  it  is  difficult  for 
students  to  relive  their  educational  experiences,  and  much  more  difficult  to  reverse  negative  consequences  of  policy 
decisions.  Even  with  the  best  intentions  of  wanting  to  improve  education,  attempts  to  change  will  be  based  mostly 
on  trial  and  error.  At  a time  in  American  education  where  our  education  systems  face  very  real  and  pressing 
problems,  it  may  appear  to  be  the  wrong  time  to  claim  that  we  need  good  educational  systems  theory.  But,  in  fact, 
we  do. 

In  the  decades  following  the  publication  of  A Nation  at  Risk  in  1983  (National  Commission  on  Excellence 
in  Education),  considerable  effort  has  been  undertaken  to  improve  public  schooling.  Reform  efforts  have  been 
typically  referred  to  as  site-based  management,  school  restructuring,  and  educational  systems  design.  Researchers 
such  as  Banathy  (1991),  Reigeluth  (1992),  Frick  (1991),  Jenlink,  Reigeluth,  Carr  & Nelson  (1996),  Caine  & Caine 
(1997),  Duffy,  Rogerson  and  Blick  (2000)  and  Senge,  Cambrom-McCabe,  Lucas,  Smith  and  Kleiner  (2000)  have 
argued  for  systemic  change  in  education.  Systemic  change  contrasts  with  numerous  piecemeal  reform  efforts  that 
have  largely  failed  in  twentieth-century  schooling.  However,  the  rhetoric  of  systemic  change  is  not  likely  in  itself  to 
make  any  real  difference  in  schooling,  since  such  rhetoric  has  been  around  for  some  time. 

For  intelligent  action,  a scientifically  based  theory  that  could  explain  and  predict  the  behavior  of 
educational  systems  is  needed.  By  this,  we  do  not  refer  to  a learning  theory,  a pedagogical  theory,  an  instructional 
method,  a leadership  theory,  classroom  management  theory  nor  a curriculum  theory  - but  an  educational  systems 
theory,  a theory  to  describe,  explain  and  predict  whole  educational  systems  and  their  transactions  with  societies  in 
which  they  are  embedded.  Educational  systems  theory  should  precisely  define  the  concepts  and  relationships  of 
system  elements  and  provide  operational  ways  in  which  these  can  be  observed  and  measured.  Education  does  not 
have  the  equivalent  of  a Newtonian  theory  of  physics.  Educators  seldom  agree  on  definitions  of  terminology.  We 
do  not  have  well  established  and  clearly  defined  terms  such  as  mass,  force,  acceleration,  velocity,  time,  gravity,  etc. 
as  in  physics.  In  short,  we  lack  a scientific  educational  systems  theory. 

Development  of  General  Systems  Theory  (GST) 

The  concept  of  general  systems  theory  (GST)  was  first  introduced  by  Ludwig  von  Bertalanffy  in  1937. 
Bertalanffy  (1968)  argued  that  there  exists  a general  theory  that  could  characterize  the  behavior  of  systems, 
regardless  of  whether  these  are  scientific,  natural  or  social;  and  he  proposed  GST  as  an  interdisciplinary  theory  that 
could  contribute  to  the  unity  of  science.  System  behavior  results  from  the  relationships  between  its  components,  and 
is  not  just  a simple  summation  of  its  parts.  The  characteristics  of  each  system  component  therefore  cannot 
adequately  explain  how  the  system  itself  behaves. 

Since  then,  there  have  been  extensive  contributions  by  others  in  the  development  of  GST  as  a logical  and 
mathematical  theory  to  provide  an  “exact  language  permitting  rigorous  deductions  and  confirmation  (or  refusal)  of 
theory”  (Bertalanffy,  1972,  p.30).  Others  have  also  contributed  well-developed  descriptive  theories  (e.g.,  Wymore, 
1967;  Cornacchio,  1972;  Mesarovic  & Takahara,  1975;  Lin,  1987;  Lin,  1999;  Bar-Yam,  2003).  In  education,  GST 
has  been  used  by  researchers  to  discuss  educational  systems  design  and  systemic  change,  but  these  approaches  have 
not  been  grounded  in  scientific  theory  about  educational  systems  (Banathy,  1991;  Caine  & Caine,  1997;  Duffy, 
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Rogerson  & Blick,  2000;  Senge,  Cambron-McCabe,  Lucas,  Smith,  Dutton  & Kleiner,  2000).  Rather  these 
approaches  largely  describe  processes  through  which  organizations  can  change,  not  whether  those  changes  are  likely 
to  result  in  desired  outcomes. 

The  SIGGS  theory  model  provided  the  first  extensive  formalization  of  a GST  model  for  educational 
theorizing  (Maccia  & Maccia,  1966;  Steiner,  1988).  Through  the  synthesis  of  four  theories:  Set,  Information,  di- 
Graph,  and  General  Systems,  SIGGS  provided  a logical  description  of  general  system  properties,  which  enabled  the 
development  of  an  educational  theory  of  school  systems  by  the  logical  process  of  retroduction  from  the  201  SIGGS 
hypotheses.  Frick,  Hood,  Kirsch,  Reigeluth,  Walcott  and  Farris  (1994)  extended  Maccia  and  Maccia’ s work  by 
classifying  the  system  properties  into  basic,  structural,  and  dynamic  properties.  This  classification  recognized  that 
some  SIGGS  properties  were  structural  as  they  described  the  connectedness  between  system  components  (SIGGS 
Website,  1996a).  Yet,  others  were  dynamic  and  described  how  patterns  of  relationships  between  system  components 
are  altered  due  to  changes  within  the  system  or  between  the  system  and  its  environment  (SIGGS  Website,  1996b). 
Thompson  (2005)  recognized  that  the  structural  properties  essentially  defined  the  system  topology. 

To  provide  a theory  that  is  logically  and  mathematically  sound,  a system-descriptive  axiom  set  is  needed. 
Although  SIGGS  was  fairly  comprehensive,  there  was  no  attempt  to  analyze  the  201  hypotheses  for  consistency  nor 
to  finalize  an  axiom  set  that  would  be  the  underlying  axioms  for  a GST.  Thompson  has  since  been  developing 
Axiomatic  Theories  of  Intentional  Systems  (ATIS),  which  is  a logico-mathematical  theory  model  for  analyzing  and 
predicting  behavior  of  systems  that  are  goal-directed  or  intentional.  Using  the  original  SIGGS  hypotheses, 
Thompson  developed  a nomenclature  to  define  system  properties,  which  improved  the  precision  with  which  SIGGS 
properties  could  be  used  (Thompson,  2005).  Thompson  also  identified  an  initial  list  of  approximately  100  axioms 
(subject  to  change,  as  this  work  is  on-going),  and  extended  the  73  SIGGS  general  system  properties  to  136  in  ATIS 
(APT&C  Website,  2005).  Development  is  on-going  and  theorems  are  now  being  derived  from  the  ATIS  axioms  for 
validation. 

SimEd  is  a software  program  designed  to  model  educational  systems.  It  is  a model  of  an  education  system 
and  is  designed  so  that  selected  parameters  can  be  evaluated  to  determine  projected  outcomes  in  view  of  these 
parameters.  Any  behavior-predictive  software  must  be  founded  on  a logical  base  of  some  kind.  The  axioms  of  ATIS 
are  being  used  as  the  rule-base  for  SimEd  in  the  development  of  educational  systems  theory. 

Using  General  System  Properties  to  Describe  an  Educational  System 

Following  are  a few  examples  of  how  general  system  properties  formulated  in  ATIS  can  be  applied  to 
educational  systems.  For  greater  detail,  the  reader  is  referred  to  extensive  reports  by  Thompson  (APT&C  Website, 
2005):  http://www.indiana.edu/~aptfrick/reports/ . 

Basic  Properties 

Basic  properties  define  the  initial  attributes  required  to  identify  and  analyze  a system.  In  ATIS,  there  are 
only  three  Basic  properties — complexness,  general  system  state,  and  size.  For  example,  a system  consists  of  at  least 
two  components  that  are  connected  by  an  affect  relation.  Understood  in  the  context  of  an  education  system,  one 
example  would  be  teachers  and  students,  who  are  components  that  are  connected  together  by  a ‘guidance  of 
learning’  affect  relationship.  These  affect  relations  determine  the  complexness  of  the  system.  The  formal  definition 
and  logico-mathematical  typology  of  ‘complexness’  is: 


Complexness,  =dfthe  connectedness  of  an  affect  relation. 

•#(§)  =df  (-amejzi)  I (M)e^lu=>  *e  S 


Complexness  is  measured  by  the  number  of  connections. 

Structural  Properties 

Strongness’  is  an  example  of  a structural  property  that  describes  relationships  between  system  components. 
‘Strongness’  is  defined  formally: 
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Strong  system  (strongness),  s&?  df  a system  with  affect  relation  sets  characterized  by  strongly 
connected  components* 


sS  1 3J4./  s&) 

‘Strongly  connected  components’  means  that  all  components  in  the  affect  relation  set  are  connected  to  each  other, 
but  at  least  one  of  the  connections  is  unilateral  (one  or  more  is  not  bi-directional;  otherwise  the  components  would 
be  completely  connected). 

Assume  that  we  are  examining  the  affect  relation,  ‘guidance  of  learning’,  in  a classroom.  If  classroom 
instruction  is  solely  from  the  teacher  (e.g.,  demonstrating,  explaining,  questioning,  prompting,  and  evaluating 
student  responses),  then  such  ‘guidance  of  learning’  is  unilateral  connectedness  from  the  teacher  to  students. 
Strongness  can  be  increased  if  there  were  more  connections  between  system  components.  For  example,  when 
students  work  in  project  groups,  ‘guidance  of  learning’  connections  can  be  created  among  students  as  they  share 
what  they  know  with  each  other.  Such  affect  relations  become  ‘completely  connected’  when  all  of  them  have  bi- 
directional connections  with  each  other.  ‘Completely  connected’  components  are  defined  formally: 

Completely  connected  components  set,  -df  a set  of  system  components  that  are  pair- wise 
path-connected  in  both  directions. 

ct®  “tit  K - M *fc9tt-50  A JifcER-L*  * 1 A Al} 


Dynamic  Properties 

‘Adaptableness’  is  an  example  of  a dynamic  property  that  describes  how  the  relationship  between  system 
components  changes  over  time.  It  is  defined  formally: 


Adaptable  system  ( a <1  apt  able  ness),  _df  a system  compatibility  change  within  certain  limits  to 

maintain  stability  under  system  environmental  change. 


aS  jr to nm lh  A < a lh  £BStoxt(2) 


For  example,  a school  system  has  high  adaptability  if  its  graduation  rates  do  not  vary  significantly  when  the 
standards  for  passing  state  examinations  are  raised. 

‘Filtration’  is  another  example  of  a dynamic  property.  It  describes  the  criteria  a system  uses  to  determine 
which  toput  qualifies  as  input  to  the  system.  The  criteria  for  selecting  its  applicants  act  as  a filter  for  entry  to  the 
school  (e.g.  students  who  are  less  than  5 years  old  are  typically  not  allowed  to  enter  K-12  schools).  ‘Filtration’  is 
defined  formally: 

Filtration,  PX&X  “lf  the  set  of  toput  system-control  qualifiers  that  control  feediw  of  toput. 

= f (P(»)  I P(»e  ry^f:  A A™' '“"'to,  Co,:  Ty  x-rT^,  ->  (T^y) 


Simulating  Change  in  Education:  An  Application  of  Logic-based  Simulation  in  SimEd 
Logic-Based  vs.  Scenario-Based  Simulation 

Scenario-Based  programs  are  defined  as  programs  that  provide  “scripts”  to  determine  outcomes.  The 
scripts  can  be  narrative  or  quantitative.  Narrative  scripts  characterize  the  qualitative  parameters  of  a system;  that  is, 
the  social,  philosophical,  and  individual  descriptions  and  the  uncertainty  of  future  outcomes.  Quantitative  scripts 
define  the  scientific  facts,  known  or  credible  data,  and  quantitative  models  that  are  used  to  determine  future 
outcomes.  However,  regardless  of  the  type  of  script,  their  content  is  closed;  that  is,  there  are  a limited  number  of 
possible  outcomes,  and  the  scripts  predetermine  the  outcomes. 

Friedman  (1999)  recognizes  this  closed  characteristic  of  Scenario-Based  Models  in  his  report  on  SimCity, 
“The  Semiotics  of  SimCity,”  when  he  states: 
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Of  course,  however  much  "freedom"  computer  game  designers  grant  players,  any  simulation  will 
be  rooted  in  a set  of  baseline  assumptions.  SimCity  has  been  criticized  from  both  the  left  and  right 
for  its  economic  model.  It  assumes  that  low  taxes  will  encourage  growth  while  high  taxes  will 
hasten  recessions.  It  discourages  nuclear  power,  while  rewarding  investment  in  mass  transit.  And 
most  fundamentally,  it  rests  on  the  empiricist,  technophilic  fantasy  that  the  complex  dynamics  of 
city  development  can  be  abstracted,  quantified,  simulated,  and  micromanaged. 

On  the  other  hand,  Logic-Based  Models  are  not  dependent  on  analyses  of  predetermined  values,  but  on  the 
logic  of  a theory  that  has  been  shown  to  be  valid  for  the  targeted  empirical  system;  for  example,  an  education 
system.  The  theory  describes  the  empirical  system  in  terms  of  its  affect  relations,  properties,  and  axioms.  The  theory 
is  then  used  to  project  outcomes  founded  on  the  theory  with  respect  to  input  parameters.  The  instantiated  axioms 
would  generate  a set  of  outcomes,  which  become  input  parameters  that  instantiate  yet  more  axioms.  Unlike 
Scenario-Based  Models  that  are  closed  due  to  the  limited  number  of  scripts,  Logic-Based  Models  potentially  have  an 
infinite  number  of  outcomes.  This  is  the  model  that  SimEd  is  based  upon. 

Example  of  SimEd  PESO  Predictions 

Suppose  a school  called  Smithtown  is  labeled  as  failing  by  NCLB  criteria,  and  consequently  student 
enrollments  begin  to  drop  (input  decreases),  as  parents  send  their  children  to  other  schools.  What  would  happen  as  a 
result  of  falling  enrollments?  How  might  SimEd  help  administrators  of  Smithtown  school  predict  and  plan  their 
required  change  of  strategies? 

Using  system  properties  defined  by  ATIS,  if  the  student  enrollments  in  the  Smithtown  School  System  are 
decreasing,  this  means  that  the  educational  systems  property,  input,  is  decreasing.  One  enters  the  current  system 
conditions  into  the  SimEd  PESO  prototype  as  illustrated  in  Figure  1.  PESO  refers  to  Predicting  Education  System 
Outcomes.  PESO  is  one  of  the  SimEd  Technologies. 


Figure  1:  If  input  is  decreasing  in  the  educational  system,  the  options  set  from  the  Educational  System  Theory  are 
applied  when  this  condition  is  true. 


Next,  SimEd  PESO  applies  the  relevant  axioms  and  theorems  from  ATIS  to  make  predictions,  as  illustrated 
in  Figure  2. 
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Figure  2:  Predictions  made  by  SimEd  PESO  when  it  is  the  case  that  educational  systems  input  is  decreasing. 


It  can  be  seen  from  Figure  2 that  the  following  axioms  have  been  triggered: 

10:  If  input  decreases,  then  fromput  decreases. 

11:  If  input  decreases,  then  storeput  decreases. 

13:  If  input  decreases,  then  filtration  increases. 

16:  If  storeput  decreases,  then  feedout  decreases. 

28:  If  filtration  increases,  then  adaptability  increases. 

How  SimEd  PESO  Makes  Logic-based  Predictions 

Even  though  there  are  approximately  100  axioms  in  ATIS  at  the  current  moment  (APT&C  Website,  2005), 
only  5 axioms  apply  under  this  condition  (input  decreases).  Axioms  10,  11  and  13  predict  the  outcomes  of 
decreasing  input.  However,  Axiom  11  predicts  a decrease  in  storeput,  which  triggers  Axiom  16.  Similarly,  Axiom 
13  triggers  Axiom  38.  This  kind  of  chaining  illustrates  how  the  inference  engine  that  is  built  into  SimEd  PESO 
works.  SimEd  PESO  actualizes  the  logical  implication  of  transitivity  - e.g.,  if  A implies  B,  and  if  B implies  C,  then 
A implies  C. 

Interpreting  the  Predictions 

It  can  be  seen  that  when  enrollments  decrease  in  Smithtown  school,  there  will  be  fewer  students  attending 
classes  (storeput  decreases)  and  fewer  students  who  will  fulfill  exit  requirements  (e.g.,  qualify  for  graduation). 

Thus,  fromput  decreases  also.  When  there  are  fewer  students  attending  classes  (storeput  decreases),  there  will  be 
fewer  students  who  eventually  leave  the  system  as  feedout  (e.g.,  through  graduation  or  as  dropouts).  Another 
prediction  is  that  filtration,  or  criteria  for  entry  into  Smithtown,  increases.  Inadvertently,  the  NCLB  rating  has  acted 
as  a filter  by  deterring  parents  from  sending  children  to  Smithtown.  While  the  axioms  discussed  may  seem  logically 
obvious,  Axiom  28  also  predicts  that  Smithtown  would  change  to  maintain  system  stability.  If  filtration  increases, 
adaptability  is  also  predicted  to  increase.  For  example,  Smithtown  school  may  try  to  change  instructional  practices 
so  that  student  performance  will  be  improved,  so  that  the  school  can  be  removed  from  the  NCLB  “failing”  list.  It 
could  also  adapt  by  firing  incompetent  teachers,  and  hiring  competent  ones  - though  this  kind  of  adaptation  would 
be  typically  resisted  by  teacher’s  unions  and  tenure  stipulations.  It  could  adapt  by  purchasing  and  using  computer 
software  to  tutor  students  who  are  not  meeting  expected  annual  yearly  progress. 
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Using  SimEd  to  Stimulate  Change 

One  way  that  Smithtown  school  could  better  its  enrollment  rate  is  to  improve  student  achievement. 

Suppose  Smithtown  wants  to  improve  its  learning  guidance  by  changing  its  predominantly  teacher-guided  system  to 
use  more  instructional  technology  and  bringing  in  parent  or  senior  citizen  volunteers  as  teacher  aides.  Essentially, 
Smithtown  is  attempting  to  increase  system  strongness  with  respect  to  instructional  affect  relations  (i.e.,  increase  the 
number  of  ‘guidance  of  learning’  connections  to  students). 

Suppose  that  an  education  system  wants  to  increase  the  system  property  ‘strongness’.  What  does  SimEd  PESO 
predict? 


Figure  3:  Educational  system  property,  strongness,  is  increasing.  What  will  SimEd  PESO  predict? 


It  can  be  seen  in  Figure  4 below  that  increasing  system  strongness  has  an  impact  on  the  hierarchical  order 
and  flexibility  of  instructional  relationships  in  classrooms.  It  will  also  affect  other  aspects  of  the  system  such  as  the 
enrollment  (input)  and  criteria  for  entering  the  system  (filtration).  Correspondingly,  these  in  turn  impact  other 
aspects  of  the  system  such  as  isomorphism  (how  the  system  replicates  the  same  strategy  to  other  parts  of  the  system) 
and  regulation  (which  fromput  qualifies  to  become  output). 

These  are  but  two  examples  of  what  the  SimEd  PESO  software  is  currently  capable  of  doing.  Through  the 
SimEd  PESO  interface,  users  will  be  able  to  predict  behavior  of  complex  systems  via  Axiomatic  Theories  of 
Intentional  Systems  (ATIS)  without  needing  to  be  experts  in  mathematics  or  logic. 

SimEd  Technologies  Will  also  Include  APT&C  Software 

Education  system  administrators  and  researchers  will  also  need  to  be  able  to  measure  system  properties 
such  as  strongness,  flexibility,  filtration,  centrality,  etc.  We  have  recently  obtained  funding  for  developing  software 
to  measure  system  dynamics  and  structure:  Analysis  of  Patterns  in  Time  and  Configuration.  APT&C  is  a different 
kind  of  measure  paradigm  that  bridges  traditional  quantitative  and  qualitative  research  methods.  APT&C  builds  on 
work  done  by  Frick  (1990)  on  APT  and  by  Thompson  (2005).  Further  information  on  APT&C  and  additional 
references  are  found  in:  http://education.indiana.edu/~frick/proposals/apt&c.pdf. 
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Figure  4:  SimEd  PESO  makes  a number  of  predictions  when  system  strongness  increases.  The  cascade  of 
inferences  is  indicated  by  successive  iterations  of  the  axioms  and  theorems. 


Conclusion 

SimEd  Technologies  are  theories,  methodologies  and  software  tools  to  help  people  understand  complexity 
in  educational  systems.  As  the  validity  of  the  Education  Systems  Theory  is  established,  educators  can  use  SimEd 
Technologies  to  better  understand  how  education  systems  can  be  improved.  A valid  educational  systems  theory  will 
show  educators  all  the  consequences  - even  the  unintended  consequences  - of  changing  one  part  of  the  complex 
educational  systems  they  direct.  These  consequences  could  then  be  pre-empted  and  managed  before  the  impact  of 
changes  is  realized. 
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Abstract 

The  purpose  of  this  research  is  to  clarify  the  details  of  social  interactions  and  experiences  players 
encounter  in  the  non- combat  Massively  Multiplayer  Online  Game  (MMOG)  environment.  The  research  focuses  on 
creating  fundamental  data  for  discussion  useful  for  implementation  for  online  learning  courses.  An  ethnographic 
study  approach  is  used  for  the  research.  Several  possible  themes  which  are  relevant  to  the  social  interaction  in  the 
environment  are  found.  Applicability  of  those  themes  to  the  educational  settings  is  discussed. 

Introduction 

Video  gaming,  today,  is  becoming  one  of  the  major  forms  of  entertainment,  and  as  the  industry  has  kept  growing 
so  has  its  market  size.  Generations  born  after  1970  are  called  “the  game  generation.”  Videogames  are  naturally  a 
major  form  of  entertainment  for  the  game  generations  (Prensky,  2003).  Online  games  provide  players  with 
opportunities  to  interact  with  others  who  are  physically  disconnected.  Among  the  genres  of  the  online  game, 
Massively  Multiplayer  Online  Game  (MMOG)  has  become  one  of  the  most  popular  genres.  The  games  in  the  genre 
have  acquired  millions  of  players  (IGDA,  2004). 

As  using  digital  games  for  instruction  has  directed  notable  interest  toward  education,  and  extensive  research  has 
been  conducted  in  this  area  (Mitchell  & Savill- Smith,  2004).  One  aspect  of  games  that  educators  and  instructional 
designers  have  focused  on  is  enhancing  the  social  interaction  in  online  learning  environments  (Muirhead,  2001). 
MMOG  is  also  becoming  a field  of  educational  research.  MMOG,  considered  a learning  community  (Steinkuehler, 
2004),  prompted  college  instructors  to  use  MMOGs  as  a virtual  classroom  (Terdiman,  2004). 

What  often  happens  when  games  are  designed  and  used  for  educational  purposes  is  that  they  lose  their  aspects  of 
fun,  and  the  game  does  not  engage  student  participation  as  one  would  expect.  For  this  reason,  principles  and 
methodologies  especially  for  utilizing  games  to  maintain  their  engagement  aspect  in  the  learning  environment  need 
to  be  further  researched  and  established. 

The  primary  focus  of  this  research  is  to  investigate  social  interaction  in  the  MMOG  environment.  In  the  MMOG 
environment,  game  players  engage  in  peer  mentoring  and  mutual  learning,  and  the  game  environment  becomes  an 
online  learning  community  (Yee,  2002b).  This  type  of  environment  is  what  educators  try  to  create  in  educational 
games. 


Literature  Review 

Social  interaction  in  online  community 

Social  interaction  is  defined  as  “the  acts,  actions,  or  practices  of  two  or  more  people  mutually  oriented  towards 
each  other's  selves,  that  is,  any  behavior  that  tries  to  affect  or  take  account  of  each  other's  subjective  experiences  or 
intentions “(Rummel,  1976).  Online  community  is  a research  topic  investigated  by  researchers  in  various  fields  (Tu 
& Corry,  2001).  It  is  also  called  a “community  in  cyberspace”  (Mccarty,  1999),  and  a “virtual  community” 
(Erickson,  1997).  Wagner  (1997)  outlined  specific  instructional  outcomes  which  enhance  learners’  experience 
achievable  through  interaction  in  distance  education  settings  and  emphasized  that  social  interaction  in  the  distance 
education  environment  is  important. 

Massively  Multiplayer  Online  Game  (MMOG)  research 

Past  research  on  MMOG  investigated  the  following  issues:  (1)  understanding  the  ways  people  are,  or  are  not, 
attracted  to  the  game,  and  game  addiction  (Yee,  2002a;  Choi  & Kim,  2004;  Foo  & Koivisto,  2004);  (2)  clarifying 
how  social  interaction  plays  a role  to  engage  people  (Steinkuehler,  2004;  Yee,  2002b);  (3)  defining  the  experience 
which  players  have  in  the  MMOG  environment  (Griffiths  et  al,  2003);  and  (4)  economic  activities  in  a virtual  world 
(Castronova,  2002). 

The  research  shows  that  social  interaction  among  the  players  takes  on  an  important  role  in  creating  an  online 
community  in  the  game  environment.  However,  most  of  the  past  research  has  been  conducted  on  combat  type 
MMOGs  such  as  “Ever  Quest”  and  “Ultima  Online,”  and  not  on  non-combat  MMOGs.  The  combat  type  games  are  a 
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type  of  “Dungeons  and  Dragons”  game,  which  contains  clear  missions  in  the  game  system  such  as  fighting  with 
monsters  and  raising  the  strength  level  of  the  character.  Even  though  the  game  players  construct  online  communities 
in  such  environments,  the  game  system  and  game  contexts  are  not  analogous  to  the  learning  communities  in 
educational  settings. 

Detailed  research,  focused  on  social  interaction  in  the  non-combat  environment  is  necessary  to  clarify  why  and 
how  social  interaction  works  for  online  game  players  in  terms  of  engagement  and  motivation.  The  research  has  to  be 
designed  to  provide  a clear  understanding  of  what  engages  players  and  what  does  not  in  the  game  environment. 
Detailed  data  on  player  behaviors  and  thoughts  need  to  be  collected,  as  well. 

Research  purpose 

The  purpose  of  the  research  is  to  describe  the  social  interactions  and  experiences  among  game  players  in  the 
non-combat  Massively  Multiplayer  Online  Game  (MMOG)  environment. 

The  research  is  aimed  at  collecting  data  to  answer  the  following  questions: 

(1)  What  are  the  elements  creating  social  interactions  which  enhance  players’  motivations  to  play  the 
game? 

(2)  What  are  the  players’  experiences  which  promote  their  engagement  in  the  game  play? 

This  research  anticipates  producing  fundamental  data  to  understand  what  kinds  of  social  interaction  occur  in  the 
MMOG  environment.  The  research  is  also  expected  to  establish  a framework  which  can  be  applied  to  other  MMOG 
research  and  other  online  community  research,  which  investigates  experiences  of  people  in  the  environment. 

The  field:  MMOG  “A  Tale  in  the  Desert” 

The  research  is  done  on  the  game  environment  of  “A  Tale  in  the  Desert”  (ATITD),  a non-combat  MMOG  which 
simulates  ancient  Egypt  world,  provides  the  players  with  a cooperative  environment  in  which  players  need  to 
collaborate  with  others  to  accomplish  their  missions  (Figure  1).  “The  objective  of  ‘A  Tale  in  the  Desert’  is  to  build 
the  ideal  civilization  by  perfecting  the  ‘Seven  Disciplines  of  Man’.  The  outcome  of  the  game  depends  solely  on  the 
players.  The  storyline  will  revolve  around  what  the  players  do.  How  they  go  about  building  their  own  cities,  how 
they  choose  their  own  leaders,  and  how  they  face  the  stranger's  challenges.  If  they  can  organize  themselves  and 
negotiate  for  what  they  need,  Egypt  will  flourish  (eGenesis,  2003).” 


Figure  1.  The  environment  of  “A  Tale  in  the  Desert” 


Methods 

As  this  research  intends  to  collect  detailed  data  and  create  “thick  description”  (Geertz,  1973)  of  culture  in  the 
field,  ethnography  (Rossman  & Rallis,  2003)  is  chosen  as  a research  approach.  Coffey  and  Atkinson’s  qualitative 
data  analysis  (1996)  is  chosen  as  a data  analysis  guideline.  Because  authentic  ethnography  requires  long-term 
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immersion  which  does  not  fit  to  the  time  span  of  the  project,  the  research  approach  is  considered  mini- 
ethnographies. 

Observations  on  game  players’  behavior,  formal  and  informal  interviews  with  game  players,  and  document 
analysis  of  reflective  journals,  written  by  the  game  players  are  conducted.  Observations  occurred  at  the  multiple 
locations  in  the  game  environment.  Face-to-face  and  in-game  chat  interviews  take  place  with  game  players. 
Document  materials  originate  with  the  official  game  website  and  fan  community  websites. 

Research  procedures 

Data  collection  occurred  at  the  beginning  of  research.  Documents  related  to  the  game  were  collected  mainly 
originated  with  the  ATITD  wiki  website  (http://wiki.atitd.net/tale2),  and  ATITD.net  (http://www.atitd.net/home/). 
Detailed  information  regarding  game  systems,  in-game  events,  and  game  player  interaction  were  compiled. 
Observations  in  the  game  environment  were  conducted  to  identify  broad  ideas  on  the  behaviors  of  the  game  players. 
Several  in-game  events  were  observed.  Interviews  sourced  from  online  chat  and  in-game  chat  functions  in  the  game. 
These  interviews  consisted  of  seven  formal  interviews  (six  face-to-face  interviews,  one  online-chat  interview). 
Interviewees  recruited  from  different  identity  groups  allowed  data  collection  from  multiple  aspects.  An  experienced 
MMOG  player  who  has  played  ATITD  for  a long  time,  a novice  game  player  who  played  ATITD,  experienced 
MMOG  game  players  who  played  ATITD  for  the  first  time,  and  a mother  whose  son  is  an  experienced  MMOG 
player,  were  the  interview  subjects. 

The  analysis  phase  included  reconstruction  of  collected  data.  The  expectation  was  that  analysis  will  also  reveal 
indications  of  interaction  patterns,  their  meanings,  power  relations,  roles,  and  interactive  sequences.  Based  on  the 
reconstructed  data,  a set  of  descriptions  and  evidences  to  addressing  the  research  purpose  and  responding  to 
questions  emerged. 


Discussion  and  Results 

Game  systems  - the  engine  for  social  interaction 

There  are  some  cultural  forms  found  through  the  data  analysis.  The  game  systems  and  those  cultural  forms  are 
closely  related.  Open-ended  environment,  cooperative  tasks,  and  user-participated  operation  create  the  cooperative, 
autonomous  player  community. 

(1)  Open-ended  environment 

There  are  certain  requirements  to  accomplish  tasks,  basically  the  game  players  are  able  to  choose  to  pursue 
any  tasks  as  they  like.  They  run  across  the  land  of  Egypt,  visit  seven  different  types  of  schools  and  universities 
(Schools/Universities  of  Leadership,  Human  Body,  Worship,  Art  and  Music,  Architect,  Conflict,  and  Thought) 
and  learn  skills  (e.g.  planting,  cooking,  mining,  and  construction)  collect  resources  (e.g.  woods,  sands,  irons,  and 
slates)  they  need,  and  produce  products  (e.g.  bricks,  carrots,  linens,  and  beers)  which  are  necessary  to 
accomplish  the  tasks. 

(2)  Cooperative  tasks 

The  game  players  are  required  to  accomplish  the  tests  assigned  by  schools  and  universities.  Some  of  the  tests 
can  be  done  individually,  however,  most  of  the  tests  require  (or  encourage)  cooperation  with  other  players.  The 
players  also  have  to  donate  resources  to  the  universities  to  release  the  higher-level  skills.  The  resources  required 
for  releasing  skills  are  large  and  difficult  to  gather  alone.  It  is  also  necessary  to  cooperate  with  other  players  to 
release  the  skills. 

(3)  User-participated  operation 

One  of  the  unique  features  of  ATITD  is  that  the  game  system  depends  highly  on  user  participation.  The 
game  websites  (e.g.  ATITD  wiki  and  ATITD.net)  are  administrated  and  updated  by  player  community.  Tests  in 
the  game  are  also  designed  to  encourage  player  participation  to  create  social  interaction  by  themselves.  To  pass 
the  test  of  the  host,  the  players  have  to  host  in-game  events.  The  events  are  such  as  a gemcutting  contest,  a 
fireworks  contest,  a cooking  contest,  and  so  on. 

ATITD  also  has  a unique  legal  system  in  which  the  players  prepare  and  pass  around  a legal  petition  to  create 
a new  game  rule.  Other  players  sign  the  petition  if  they  agree  to  the  new  rule.  Once  the  law  is  approved,  the 
request  is  sent  to  the  game  designers.  For  example,  “Driftwood  Recycling  Act”  is  the  law  to  require  the  players 
on  the  novice  island  to  remove  the  constructions  they  created,  and  “Full  ban  of  Machiavelli”  is  the  law  to  exile  a 
misbehaving  player.  These  laws  petitioned  by  the  players  improve  the  quality  of  community  life.  Game 
publisher  usually  handle  these  sorts  of  requests  as  customer  requests,  however,  ATITD  includes  the  request 
process  as  a part  of  its  game  system. 

In-game  events  take  place  to  increase  entertainment  in  the  MMOG  world.  Though  the  game  publisher 
usually  produces  the  events  in  other  MMOGs,  in  this  game,  players  are  also  able  to  host  the  events.  In-game 
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events  are  opportunities  to  meet  with  other  players.  Just  like  in  the  real  world,  people  enjoy  conversation  and 
play  with  the  skills  they  learned.  “. . .Others  plan  it  in  advance  and  invite  a lot  of  people,  then  [sic]  have  drinks 
and  aero  and  games.  :)”  (Data  from  online-chat  log,  04/17/2005) 

Guilds  - the  place  of  mentorship 

A guild  is  a player  group  to  share  buildings,  resources  and  manpower.  It  also  works  as  a base  for  the  players. 

The  guild  has  a rank  system  and  the  higher-ranked  players  manage  the  guild.  In  the  guild  elder  players  support 
novice  players  through  mentorship,  or  apprenticeship  style  instruction.  The  following  dialogue  is  from  online  chat  of 
the  guild  channel.  A novice  player  learned  how  to  move  to  other  areas  using  a chariot  stop  from  a guild  elder. 

A:  How  I can  travel  to  other  area? 

B:  ok,  there  are  chariot  stops,  which  are  also  shown  on  your  map.  They  go  to  different  regions.  You  can  find 
what  goes  where  on  wiki 

B:  eventually  you  can  get  anywhere  :)  [smiley  face] 

B:  the  chariot  stop  is  up  the  road  from  Thoth,  north. . .from  the  chariot  stop  you  would  go  to  Nile  Delta,  then 
Sinai,  then  RSO.  Travel  at  the  chariot  stop  can  be  free  if  you  want  to  wait,  or  if  you  are  in  a hurry,  you  can  use 
travel  time,  it  will  tell  you  when  you  clock  [click]  on  it. 

A:  Cool,  now  I can  see  pyro 

B:  :)  (Data  from  the  online-chat  log,  03/27/2005) 

“Aero?”  - the  designed  social  interaction  patterns 

As  the  game  designers  of  ATITD  intended,  social  interaction  opportunities  are  designed  and  embedded  in  the 
game  play.  The  players  experience  various  types  of  social  interaction.  One  of  the  unique  phenomena  observed  is  that 
the  players  start  communicating  with  other  players  by  asking  “aero?”  It  means  “Would  you  like  to  play  acrobatics 
with  me?”  Acrobatics  is  one  of  the  tests  for  the  Human  Body  in  the  game.  The  players  have  to  learn  28  moves  to 
pass  the  test.  To  learn  the  moves,  the  players  stand  close  to  each  other  and  perform  the  moves.  The  players  need  to 
communicate  with  others  to  know  if  they  know  the  other  moves. 

KK:  I have  a petition  that  needs  sigs,  if  anyone’s  within  reach?  [petition  has  to  be  signed  by  other  players  to  be 
approved] 

P:  KK,  I will  check  it  out  after  the  competition 
KK:  Didn’t  you  take  a copy,  P? 

(Data  from  the  online-chat  log,  04/09/2005) 

Barrier  created  by  the  social  interaction 

On  the  other  hand,  existence  of  other  player  may  create  a barrier  for  the  new  players.  Below  is  dialog  with  a 
novice  ATITD  player  who  has  had  no  experience  of  MMOG  play.  She  reflected  why  she  did  not  feel  engaged  in  the 
game. 

Q:  . . .So  you  are  saying  you  have  the  same  personality  in  the  real  and  in  the  game. 

A:  Absolutely.  I tell  you  what,  not  in  the  game,  in  the  real  world,  if  I am  in  the  event,  there  is  one  or  two 
persons  I know,  then  I am  fine.  But  when  there  are  no  people  I know,  then  I become  uncomfortable,  and  I will 
find  a reason  to  leave. 

Q:  Yeah,  I see...  In  the  tale  in  the  desert,  did  you  talk  to  the  mentor? 

A:  Yeah,  my  relationship  with  mentor  was  uncomfortable  because  I don't  know  who  the  mentor  was.  If  the 
mentor  was  you,  or  other  members  of  the  class,  I would  have  been  fine.  But  when  I was  not  knowing  who  the 
mentor  was,  I was  not  comfortable.  (Data  from  interview  with  GS  04/1 1/2005) 

Novice  players  tend  to  be  overwhelmed  by  the  complex  MMOG  environment.  The  interviewee  above  was  also 
not  familiar  with  the  anonymity  of  the  environment.  Some  informants  stated  that  they  prefer  to  play  solo  if  their 
friends  in  the  real  world  do  not  play  the  game.  The  personality  of  the  player  seems  to  affect  his/her  attitude  to  other 
players  in  the  game. 


Conclusion  - implications  for  online  course  design 

The  social  interactions  in  ATITD  are  carefully  designed  and  embedded  in  the  game  play.  Interactions  take  place 
in  various  ways.  The  significant  differences  between  the  player  community  in  ATITD  and  learning  community  in 
online  courses  are  that  the  role  of  provider,  and  the  relationship  of  provider  and  receiver.  The  learning  community  in 
online  courses  is  usually  instructor-led.  Automated  course  are  unlikely  to  create  interaction  among  learners.  Provider 
(instructional  designer  and  instructor)  and  receiver  (learners)  are  clearly  separated.  Mutual  learning  among  the 
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learners  passively  occurs  in  partial  ways.  Continuous  facilitation  by  the  instructor  is  necessary  to  sustain  the  learning 
activities  in  the  course. 

In  ATITD,  on  the  other  hand,  the  game  designer  is  the  service  provider  and  designs  and  implements  the  game 
system  to  realize  the  planned  activity  (Figure  2).  Once  the  activity  is  implemented,  the  players  operate  the  activity 
and  the  advanced  players  voluntarily  support  novice  players  to  learn  and  participate  in  the  process. 


© — 

Game  designer 


Figure  2.  Relationship  in  the  game  world 

In-game  events  and  festivals  are  planned  and  hosted  by  advanced  player  who  pursue  the  higher-level  task.  The 
community  works  autonomously,  and  the  players  learn  mutually  without  direct  help  or  intervention  of  the  providers. 
The  main  providers’  roles  are  an  initial  design  and  implementation,  and  maintenance.  The  design  concept  and  design 
constructs  are  different  between  ATITD  and  online  courses.  Social  interaction  opportunities  are  interwoven  in 
ATITD,  whereas  social  interaction  is  not  necessarily  a concern  of  design  in  the  online  courses.  Perhaps  taking  these 
differences  into  account  in  the  online  course  design  could  improve  them. 

Playing  a new  MMOG  is  analogous  to  starting  a new  life  in  a foreign  country.  Significantly  a researcher  who 
studies  MMOG  has  to  become  acclimated  not  only  the  language,  but  also  the  culture  and  rules  of  the  “world”. 
Despite  the  hardships,  meet  the  people  in  ATITD  world  was  a wonderful  experience.  The  elders  of  the  guild  were 
generous  and  cooperative.  People  in  the  gemcutting  contest  welcomed  and  encouraged  the  novice  player.  Everyone 
encountered  during  the  research  positively  supported  each  other  to  build  a better  society.  Compared  with  other 
popular  MMOGs,  ATITD  is  just  like  a small  village  with  just  about  1500  villagers;  however,  with  a unique  and 
valuable  culture  in  the  village.  As  past  ethnographers  who  studied  uncivilized  tribes  become  enchanted  with  the 
field,  the  field  of  ATITD  provided  the  researcher  with  engaging  and  invaluable  experiences.  Amazingly  the  game 
arose  from  a small  team  of  game  developers,  but  the  encouraging  news  for  instructional  designers  is  that  a small 
team  could  create  such  an  online  learning  community  which  is  filled  with  social  interaction  and  learning 
opportunities. 
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Introduction 

This  paper  reports  on  the  findings  of  a large  qualitative  dissertation  study  of  eight  Arizona  elementary 
classrooms  in  seven  public  schools.  The  study  revealed  an  uncritical  use  of  Accelerated  Reader  (AR).  AR  software 
use  was  largely  claimed  as  exceptional  “technology  integration”  by  educators  who  believed  that  AR  enhanced 
students’  standardized  test  scores.  Such  reliance  on  technology  to  manage,  assess,  and  track  students  allowed  for 
production  and  documentation  of  students’  progress  with  “computerized  reports”,  a hallmark  of  efficiency, 
modernity,  and  preparation  for  life  in  an  industrialized  society. 

Conceptual  Framework 

The  process  of  using  technology  to  enhance  learning  is  by  no  means  a mere  technical  matter.  The 
computer,  like  the  chalkboard  and  the  textbook,  functions  in  a social  environment  that  is  mediated  by  humans. 
Educational  resources  that  make  use  of  emerging  technologies  have  not  become  the  norm  in  US  public  schools.  The 
issues  of  how  to  educate  teachers  in  the  use  of  technology  for  the  enhancement  of  learning  and  how  the  structures  of 
schooling  intersect  with  the  ideas  of  technology  integration  are  complex.  Even  as  the  potential  to  fundamentally 
reform  education  with  the  help  of  emerging  technologies  exist  (National  Research  Council,  2000),  established 
institutional  forms  of  schooling  have  been  remarkably  stable  and  resistant  to  change  (Cuban,  1993;  Cuban,  2001; 
Cuban,  Kirkpatrick,  & Peck,  2001). 

Many  scholars  contend  that  the  attempt  to  reform  education  practices  with  the  introduction  of  technology  is 
to  democratize  access  to  knowledge  and  to  give  students  the  power  of  owning  their  learning.  Popkewitz, 

Tabachnick,  and  Wehlage  (1982)  found  that  reform  programs  have  unforeseen  consequences  and  hold  hidden  values 
that  work  against  the  ethical  and  political  intentions  of  the  reform  planners.  They  establish  that  the  institutional 
values  of  schools: 

. . . are  justified  and  supported  by  professionals  in  the  schools  and  school  districts,  who  view 
them  as  responsive  to  the  values  of  the  surrounding  communities.  Teachers’  perceptions  of 
community  social  and  cultural  backgrounds  give  credence  to  technical  definitions  of 
education.  In  addition,  what  parents  expect  of  the  schools  supports  the  emphasis  on  skill 
development  and  social  control,  (p.  79) 

They  make  a case  for  how  the  activities  of  schools  historically  condition  people’s  ways  of  giving  form  to  their  social 
world.  The  intent  to  reform  schools  with  technology  may  have  unanticipated  outcomes  that  come  into  play  in  the 
reform  implementation  as  observed  in  the  practice  of  technology  in  this  study. 

Although  one  of  the  intents  of  school  reform  with  technology  was  to  give  students  the  power  of  owning 
their  own  learning,  the  observed  integration  of  technology  in  this  study  primarily  took  the  form  of  skill 
development — keyboarding  and  the  improvement  of  teacher  and  student  efficiency — computer  use  to  manage 
students’  reading  and  mathematics  education.  Such  uses  of  technology  are  far  removed  from  the  idea  of  knowledge 
construction  as  a process  of  continuous  research  and  design  in  which  students  and  teachers  share  the  difficulties  and 
results.  The  failure  of  the  school  technology  reform  effort  cannot  be  laid  upon  teachers.  In  this  study,  we  found  that 
the  organizational  structure  of  the  district,  the  school,  and  the  classroom  have  shaped  teachers’  primary  practices. 

Purpose  of  Study 

We  wanted  to  study  classrooms  that  were  typical,  i.e.,  contained  one  to  five  computers  which  seems  to  be 
the  norm  in  US  public  schools  (Becker,  2000).  In  addition  we  wanted  to  find  a school  district  known  for  its 
investment  in  instructional  technology  support.  The  school  district  we  selected  was  reputed  to  be  the  best  in  the  state 
of  Arizona  for  its  instructional  technology  support  program  and  had  replaced  all  older  Macintosh  computers  in  its  25 
primary  schools  with  new  ones  at  the  time  of  this  study  (2001-03). 
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We  hoped  that  a study  of  locally  identified  effective  computer  technology  use  practices  where  educators 
function  under  typical  conditions — one  to  five  computers  in  the  classroom  and  periodically  scheduled  access  to  the 
school  computer  lab  may  reveal  particular  attempts  at  technology  integration.  The  primary  objective  was  to  study 
teachers’  computer  technology  use  within  the  particular  context  of  their  classrooms  and  schools.  The  secondary 
objective  was  to  situate  these  practices  in  the  context  of  the  research  community’s  expectations  for  educational 
technology  and  recommended  exemplars  of  technology  use  in  education. 

Data  Sources  and  Methods 

The  research  questions  guided  the  study,  resulting  in  a qualitative  research  design  using  the  interpretive 
approach  outlined  by  Erickson  (1986).  The  inherent  complexity  of  classroom  life,  multiple  perspectives  and  the 
pluralism  of  educators’  practices  of  computer  technology  uses  within  the  context  of  their  schools  were  sought.  At 
the  same  time,  balance  and  variety  were  important;  no  less  than  five  and  no  more  than  ten  elementary  classroom 
teachers  with  wide-ranging  teaching  experiences  within  a large  school  district  were  preferred  for  participation  in  the 
study. 

Stake  (1995)  said  that  a case  is  an  integrated  system  with  people  and  interactions,  and  when  it  comes  to 
selecting  cases  the  first  criterion  should  be  to  maximize  what  can  be  learned.  The  choice  of  a case  should  lead  to 
important  understandings  and  warranted  assertions.  The  essential  question  was  what  group  of  schools  will  help  us 
understand  primary  teachers’  technology  use  practices  within  the  context  of  the  selected  school  district?  For 
collective  case  studies,  site  selection  based  on  opportunity  to  learn  is  of  primary  importance  (Stake,  1995,  p.  6).  The 
qualitative  research  attempt  is  a process  of  deliberate  inquiry  in  a specific  setting  (Bogdan  & Biklen,  1998); 
therefore  the  contexts  in  which  the  participants  are  to  be  studied  have  to  be  associated  with  the  study’s  objectives. 

Long-term  data  collection  occurred  during  the  2001-02  and  2002-03  school  years  in  eight  classrooms  in 
seven  schools  of  a large  district  with  25  elementary  (K-6)  schools  and  over  25,000  students.  Participants  were 
nominated  by  school  administrators  as  successful  in  their  educational  use  of  computers.  Eight  teachers  participated. 
The  goal  was  to  access  the  quality  of  teachers’  and  students’  interactions  with  computer  technology.  Long-term 
observations  in  classrooms  and  computer  laboratory  settings,  interviews  with  principals,  teachers,  and  technology 
specialists,  and  document  collection  were  conducted.  Teachers  completed  a technology  checklist  and  a survey  of 
their  teaching  philosophy  and  computer  use  practices.  Students  completed  a survey  of  their  access  to  and  use  of 
computer  technology  at  home.  Detailed  descriptions  of  observed  and  perceived  uses  of  computer  technology,  with 
interpretive  commentary  were  described  through  individual  case  studies. 

Site  Selection 

Researchers  make  decisions  throughout  their  studies  starting  with  the  very  important  and  yet  basic 
questions  of  “what  should  I study?”  and  “where  should  I study?”  Factors  related  to  the  school  environment  may 
promote  educators’  use  of  computer  technology  to  enhance  learning  (Becker,  1994;  Hadley  & Sheingold,  1993).  For 
that  reason,  it  was  essential  to  identify  a school  district  that  had  created  a supportive  environment  for  educators’  use 
of  computer  technology.  Awan  Public  Schools,  a large  suburban  Arizona  district  that  had  made  an  effort  to  support 
the  use  of  computer  technology  in  education  was  identified.  In  2001-02,  Awan  Public  Schools  was  the  only  school 
district  in  Maricopa  County  that  had  a technology  personnel  support  program  to  assist  classroom  educators  and 
students  with  the  instructional  use  of  computers  in  school  computer  labs.  Each  technology  specialist,  who  was  also  a 
certified  teacher,  served  two  school  campuses  and  spent  approximately  9-10  days  per  month  at  each  campus  of 
which  35  hours  was  to  be  spent  on  direct  instruction  in  the  lab  and  in  supporting  classroom  teacher’s  use  of 
technology.  This  program  was  unique  and  represented  the  administration’s  effort  to  support  the  use  of  computer 
technology  in  education  in  more  than  a symbolic  way. 

School  Site  Settings 

The  seven  primary  (K-6)  school  sites  in  the  study  functioned  under  somewhat  similar  support  structures 
organized  by  the  Awan  school  district’ s administration.  Each  school  had  one  school  administrator — the  principal, 
and  was  supported  by  district  staff  with  regard  to  technology  support  and  maintenance.  A central  computer 
laboratory  with  30  Macintosh  workstations,  a teacher  workstation,  a video  projector,  and  one  or  more  laser  printers; 
and  one  Internet  accessible  computer  in  each  classroom  was  the  standard  technology  model  at  all  the  schools.  To  a 
large  extent,  schools  in  the  study  served  middle  and  upper-middle  class  families;  the  student  population  was  not  very 
diverse. 
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Participant  Selection 

Once  the  school  district  was  identified  and  permission  was  obtained,  the  Assistant  Superintendent  for 
Curriculum  and  Instruction  introduced  the  researchers  to  all  the  primary  school  administrators  via  email.  The 
researchers  contacted  the  school  principals  by  phone  and  email,  and  asked  them  to  nominate  5th  grade  classroom 
teachers  whom  they  identified  as  successful  users  of  computer  technology  in  education.  As  the  nominations  were 
received,  researchers  spoke  with  the  nominees  and  scheduled  to  meet  with  them.  In  two  separate  instances,  the 
school  administrators  nominated  a fourth  grade  and  a fifth/sixth  grade  teacher  for  participation  in  the  study,  citing 
that  these  classroom  teachers  were  the  most  successful  users  of  computer  technology  at  their  school  sites. 

Five  female  and  three  male  classroom  teachers  participated  in  the  study;  their  teaching  experience  ranged 
from  3 to  24  years  (Table  1).  Reasons  citied  by  school  administrators  for  recommending  these  teachers’  for 
participation  in  the  study  included  their  willingness  to  use  computers,  use  of  software  programs  to  manage  students’ 
reading  and  mathematics  education,  and  use  of  computers  by  students  to  create  demonstrations  of  learning  (e.g., 
HyperStudio). 

Table  1 Comparison  of  Teacher  Experience,  Demographics,  and  Technology  Resources,  Participant  Classrooms, 


2002-03 


Descriptor 
Classroom  Teacher 

Pule 

Mr. 

M 

Bain 
Mr.  C 

Mr.  S 

Hoyt 
Mrs.  H 

Rian 
Mrs.  C 

Sloan 
Mrs.  T 

Ely 
Mrs.  L 

Oden 
Mrs.  G 

Years  of  Teaching  Experience 

24 

4 

3 

6 

12 

21 

7 

3 

Highest  Degree  Attained 

MA 

MA 

MA 

BA 

MA 

MA 

BA 

MA 

Additional  University  Credits 

72 

30 

30 

15 

60 

50 

15 

33 

Grade  Taught 

4 

5/6 

5 

5 

5 

5 

5 

5 

Class  Size 

28 

21 

28 

25 

28 

29 

28 

25 

Number  of  Limited  English 
Proficient  Students 

0 

17 

5 

4 

4 

1 

0 

0 

Number  of  Internet 
Accessible  Computers 

1 

1 

1 

2 

1 

2 

2 

1 

Total  Instructional  Use 
Computers 

14 

6 

1 

2 

2 

2 

2 

1 

Data  Analysis 

Making  sense  of  the  data  began  early  on  during  field-work.  As  a human  interpreter  of  the  observed 
phenomenon,  thoughts  about  happenings  in  short  handwritten  notes  were  documented,  ideas  about  observations 
were  scribbled  on  the  margins  of  the  field  notes  or  spoken  into  the  tape-recorder  as  the  researcher  prepared  to  leave 
a school  site  after  an  observation  occasion.  These  notes  became  a part  of  the  sense-making  process  and  formed  an 
integral  part  of  the  data  corpus.  These  early  notes  represented  the  first  interpretations  of  the  observed  events. 

Erickson  (1986)  used  a mode  of  analysis  that  can  be  described  as  analytic  induction.  Using  this  mode  of 
interpretive  research,  the  research  data  record  was  read  as  a whole  several  times  and  assertions  were  generated. 
These  preliminary  assertions  were  propositions  or  conclusions  based  on  the  data  as  a whole  that  the  authors  believed 
to  be  true.  Attempting  to  understand  the  classroom  use  of  technology,  preliminary  assertions  were  created.  Then  the 
assertions  were  warranted  by  looking  through  the  data,  seeking  to  verify  or  support  and  also  seeking  to  disconfirm 
the  assertions.  In  order  to  warrant  the  assertions,  the  data  corpus  as  a whole  was  searched  for  all  instances  of 
computer  use.  All  cases  of  computer  use  that  supported  the  particular  assertion  to  be  warranted  were  identified.  Any 
discrepant  cases  found  in  the  instances  of  computer  use  that  differed  from  that  of  the  preliminary  assertion  were 
searched,  identified,  and  examined  for  plausibility  and  consistency  of  regular  practice  of  computers  in  education. 

School  Goals  and  Technology 

From  school  mission  statements  and  stated  academic  goals  (Arizona  Department  of  Education,  School 
Report  Cards,  2002,  2003,  http://www.ade.state.az.us/srcs/,  Accessed  May  20,  2003),  it  was  evident  that  student 
achievement  and  a focus  on  reading,  writing,  and  mathematics  were  high  on  the  list  of  school  objectives.  The  school 
report  cards  which  contain  information  reported  by  school  administrators,  typically  reported  four  academic  goals. 
Based  on  the  order  in  which  the  academic  goals  were  presented,  researchers  assigned  them  a priority  rating  of  1 
(highest)  through  4 (lowest)  as  shown  in  Table  2.  The  relationship  of  the  stated  academic  goals  to  technology  uses 
and  the  expectations  for  student  learning  can  be  inferred  from  these  data.  Specific  mention  of  the  student 
achievement  was  mentioned  as  follows:  (a)  SAT  9 — Stanford  9 Achievement  test;  (b)  AIMS — Arizona  Instrument 
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to  Measure  Standards,  the  state  mandated  test  administered  in  reading,  writing,  and  mathematics  in  grades  3,5,8, 
and  high  school;  and  (c)  Six  Trait  Writing-the  Northwest  Regional  Educational  Laboratory  writing  assessment 
model  that  included  the  following  characteristics  of  writing:  ideas,  organization,  voice,  word  choice,  sentence 
fluency,  and  conventions.  All  schools  in  this  study  had  formally  prioritized  improving  student  achievement  as 
measured  by  standardized  tests.  We  found  that  these  goals  were  reflected  in  the  observed  technology  practices. 

Table  2 Comparison  of  Academic  Goals  and  Relationship  to  Technology,  Elementary  Schools  in  Study,  2002-03 

Academic  Goals Pule  Bain  Hoyt Rian Sloan  Ely Oden 

Staff  development  in  classroom  technology  1 — — — — — — 

uses 

Incorporate  / maintain  the  following  programs  in  the  curriculum 


Accelerated  Reader  (Reading) 

2 

— 

1 

— 

— 

3 

— 

Accelerated  Math  (Supplement  Instruction) 

— 

— 

— 

— 

— 

2 

— 

Increase  student  achievement  (SAT  9, 
AIMS) 

3 

— 

2 

1 

2 

4 

4 

Staff  development  in  Six  Trait  Writing 
(AIMS) 

— 

4 

— 

— 

4 

1 

— 

Implement  the  following  into  student  learning  activities 

Critical  thinking/questioning  strategies 

— 

1 

3 

— 

— 

— 

— 

Cooperative  group  work 

— 

1 

— 

— 

— 

— 

— 

Make  learning  stimulating  and  challenging 

— 

3 

— 

— 

— 

— 

— 

Apply  math  to  real-world  experiences 

— 

— 

— 

— 

— 

— 

2 

Note.  The  symbol  — is  used  to  denote  that  the  participant  school  did  not  prioritize  the  corresponding  academic  goal. 
The  number  1,  2,  3,  or  4,  represents  the  priority  of  the  academic  goal,  1 representing  the  highest  and  4 the  lowest 
priority. 

School  Computer  Lab  Model 

Schools  in  Awan  had  adopted  the  school  computer  lab  model  for  technology  integration.  This  model 
allowed  for  maximization  of  student  computer  use  in  a central  location;  however,  it  did  not  provide  access  to 
computers  in  the  one  location  where  students  spent  most  of  their  school  day — the  classroom.  The  typical  classroom 
in  participant  schools  had  1 or  2 Macintosh  computers  which  did  not  make  for  a technology-rich  classroom.  Table  1 
shows  the  number  of  student  accessible  computers  in  participating  classrooms.  The  exceptions  to  school  district, 
state,  and  national  trends  were  Mr.  M’s  fourth  grade  classroom  at  Pule  Elementary  and  Mr.  C’s  fifth-sixth  grade  dual 
language  classroom  at  Bain  Elementary.  These  exceptions  are  attributed  to  the  classroom  teachers’  personal  efforts 
to  find  additional  computers  for  classroom  use  and  not  to  school  district  policy. 

Prevalent  Software  Programs 

The  implementation  of  the  Renaissance  Learning  programs  (http://www.renleam.com,  Accessed  May  20, 
2003)  and  other  software  used  in  the  computer  lab  or  participant  classrooms  is  shown  in  Table  3.  Implementation  or 
maintenance  of  the  Accelerated  Reader  (AR)  and  Accelerated  Math  (AM)  programs  which  had  computer  software- 
based  reading  comprehension  and  math  ability  assessments  was  mentioned  as  a priority  academic  goal  by  three  of 
the  seven  schools.  Where  these  programs  were  available,  they  were  usually  used  in  the  classroom  setting.  There, 
students  took  turns  to  take  the  relevant  AR  reading  comprehension  quiz  after  reading  a book  or  scanned  their  answer 
sheets  for  the  AM  practice  and  tests  of  math  skills.  From  the  field-work  at  the  seven  school  sites,  it  was  noted  that 
Oden  Elementary  was  the  only  site  that  had  neither  implemented  nor  initiated  the  integration  of  the  AR  or  AM 
programs  into  its  school  curriculum. 

Software  programs  such  as  Kid  Pix , HyperStudio , and  Apple/C laris  Works — which  contained  word 
processing,  spreadsheet,  and  database  programs  were  typically  used  during  the  scheduled  sessions  in  the  computer 
lab.  In  some  participant  classrooms,  if  students  had  not  completed  their  research  at  home  or  did  not  have  access  to  a 
computer  at  home  and  needed  extra  time  beyond  the  regularly  scheduled  session  (once  every  6 days  for  one  class 
period)  in  the  computer  lab,  students  were  occasionally  allowed  to  use  the  Internet  for  researching  information  to 
include  in  their  social  studies  reports  (e.g.,  the  fifth  grade  report  on  selected  US  Presidents). 

Computer-Based  Programs  and  Student  Achievement 

Interest  in  improving  student  achievement  as  measured  by  standardized  tests  was  expressed  by  all  school 
principals  in  the  study.  During  formal  interviews,  most  of  the  principals  expressed  their  belief  that  use  of  certain 
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software  programs  such  as  the  Renaissance  software  programs  (e.g.,  Accelerated  Reader,  Accelerated  Math,  and 
Math  Facts  in  a Flash)  helped  increase  student  achievement.  They  believed  that  the  use  of  the  Accelerated  Reader 
(AR)  program  helped  increase  student  interest  in  reading.  They  also  expressed  their  faith  that  the  AR  program 
provided  the  all  important  reading  practice  that  helped  improve  comprehension  and  reading  ability,  thereby  helping 
to  raise  student  achievement  scores.  In  the  schools  where  the  AR  program  was  implemented,  students’  were 
assessed  at  the  start  of  the  school  year  using  the  STAR-Reading  computer-based  test  to  identify  their  reading  ability 
and  at  the  end  of  each  quarter.  Based  on  this  evaluation,  students  read  books  at  their  “zone”.  For  each  book  read, 
individual  students  took  a computerized  comprehension  quiz  (comprised  of  5 to  10  multiple  choice  questions),  and 
received  scores  and  points  as  they  completed  the  quizzes  successfully.  The  AR  software  program  kept  track  of  the 
books  read,  and  the  scores  and  points  received  for  each  student  during  the  school  year.  In  the  schools  where  AR  was 
“implemented”  (see  Table  3),  the  principals  set  school  goals  for  the  number  of  AR  points  to  be  achieved  by  students. 
The  principals  regularly  monitored  each  classroom’s  progress  in  reading  as  reported  by  weekly  or  monthly  status 
reports  generated  by  the  AR  software. 


Table  3 Comparison  of  Prevalent  Software  Programs,  Elementary  Schools  in  Study,  2002-03 


Software 

Program 

Pule 

Bain 

Hoyt 

Rian 

Sloan 

Ely 

Oden 

Accelerated 

Reader 

Implementeda 

Usedb 

Implemented 

Implemented 

Implemented 

Implemented 

Accelerated  Math 

— 

— 

Implemented 

— 

Implemented 

Implemented 

— 

Math  Facts  in  a 
Flash 

Introduced0 

Introduced 

Kid  Pix 

Used 

— 

— 

— 

Used 

— 

— 

HyperStudio 

Used 

Used 

Used 

— 

— 

— 

Used 

Claris/Apple 

Works 

Used 

Used 

Used 

Used 

Used 

Used 

Used 

Note.  The  symbol  — denotes  a participant  school  where  the  corresponding  program  was  not  implemented,  used,  or 
introduced. 

Tmplemented  = The  school  had  two  or  more  years  of  experience  in  using  the  Renaissance  programs  of  Accelerated 
Reader  or  Accelerated  Math;  the  programs  were  used  regularly  and  the  school  principal  paid  attention  to  student 
progress  in  reading  via  periodic  reports. 

bUsed  = The  school  used  the  software  program  routinely,  but  did  not  give  special  attention  or  credence  to  the 
programs;  they  were  used  generally  in  the  school  computer  lab  and  seldom  in  classroom  conditions, 
introduced  = The  school  had  recently  piloted  the  program  with  the  intention  of  integrating  the  program  into  the 
Math  education  curriculum  on  a school- wide  basis. 


Among  the  seven  school  sites,  six  schools — Pule,  Bain,  Hoyt,  Rian,  Sloan,  and  Ely  Elementary  — had 
integrated  the  Accelerated  Reader  (AR)  program  into  their  curriculum  to  varying  degrees.  In  some  school  sites, 
especially  the  ones  where  AR  was  “implemented”  vs.  “used”  (see  Table  3)  the  importance  assigned  to  AR  was 
obvious.  As  soon  as  one  walked  into  the  school,  near  the  office  one  could  find  a large  display  that  indicated  the 
school’s  monthly  or  quarterly  progress  in  AR  points  to  reach  the  school  goal.  In  2002-03,  many  schools  in  the  Awan 
School  District  that  had  successfully  integrated  the  Accelerated  Reader  program  in  previous  years  began  the 
integration  of  the  Accelerated  Math  software  program  or  another  math  practice  and  test  program  called  Math  Facts 
in  a Flash,  also  a Renaissance  software  product. 

Hoyt,  Sloan,  and  Ely  Elementary  schools  had  one  year  or  more  experience  in  integrating  the  Accelerated 
Math  (AM)  program  into  the  school’s  curriculum.  At  the  start  of  the  school  year,  students  were  tested  using  the 
computer-based  STAR-Math  assessment  which  allowed  them  to  be  placed  into  grade  level  ability  such  as  4.6  (4th 
grade,  6th  month).  Based  on  student  performance  in  the  STAR-Math  test,  the  AM  software  program  generated 
customized  practice  and  test  assignments.  Students  printed  these  software  generated  practice  assignments  and  tests, 
marked  their  answers  on  bubble  sheets,  and  self  scanned  the  bubble  sheets  to  a computer  in  the  classroom  using  a 
customized  scanner.  Student  performance  on  assignments  and  tests  were  tracked  by  the  software.  Teachers  had  the 
ability  to  set  goals  for  individual  students  in  the  software  if  they  noticed  students  struggling  or  excelling  in  specific 
skill  areas.  Every  quarter,  students  were  tested  on  the  STAR-Math  test  to  determine  their  grade  level  gains. 
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State  Legislation  on  School  Accountability 

In  2002,  the  Arizona  Department  of  Education  developed  the  “Arizona  Learns”  formula  based  on  student 
performance  on  the  Stanford  9 (SAT  9)  and  the  Arizona  Instrument  to  Measure  Standards  (AIMS)  in  response  to 
state  “school  accountability”  legislation  (http://www.azleg.state.az.us/ars/15/00241.htm,  Accessed  January  20, 

2003).  The  legislation  was  introduced  during  Fall  2001  (when  data  collection  for  this  study  started)  and  was  passed 
in  Spring  2002.  In  October  2002,  the  formula  assigned  one  of  four  possible  profiles — Excelling , Improving , 
Maintaining  Adequate  Performance , or  Underperforming — to  every  Arizona  public  school.  All  of  the  schools  that 
participated  in  the  study  received  the  second  best  profile — Improving,  only  two  schools  in  the  entire  state  received 
the  best  profile  of  Excelling.  Nevertheless,  under  such  state  school  accountability  conditions,  it  was  no  surprise  that 
school  administrators’  expressed  their  interest  in  using  supplemental  computer-based  programs  to  improve  student 
achievement  as  measured  by  the  national  level  Stanford  9 and  the  state  mandated  AIMS. 

Reading  Renaissance  Program  as  Primary  Use  of  Computers  in  the  Classroom 

A common  theme  among  the  participants’  use  of  computer  technology  in  the  classroom  was  the  use  of  the 
Reading  Renaissance  Accelerated  Reader  (AR)  program.  The  AR  program  provided  a software  management  tool  to 
test  students’  reading  comprehension  via  computer-based  quizzes.  At  the  start  of  the  school  year,  students  were 
tested  using  the  STAR  Reading  program — a computer-based  software  program  to  identify  their  reading  ability  and 
reading  zone  known  as  Zone  of  Proximal  Development  (ZPD).  Any  book  in  the  school  library  for  which  the  school 
owned  quizzes  was  known  as  an  AR  book.  On  completion  of  reading  an  AR  book  in  their  recommended  ZPD  range, 
students  took  a related  quiz  within  24  hours.  They  earned  AR  points  based  on  the  number  of  correct  answers  in  the 
quiz  of  5 to  10  multiple  choice  questions.  Students  spent  15  to  20  minutes  each  week  taking  quizzes  on  the  computer 
and  read  AR  books  for  60  minutes  each  day.  Teachers  and  administrators  monitored  students’  reading  progress  by 
reviewing  diagnostic  reports  generated  by  the  AR  software.  Teachers  intervened  when  necessary  (either  when 
students  were  struggling  or  needed  to  be  pushed  further)  by  suggesting  that  students  read  at  a different  reading  level. 
School  goals  were  set  for  the  number  of  AR  points  to  be  achieved,  school  progress  in  reading  as  measured  by  AR 
points  were  constructed  and  displayed  near  the  school  front  office.  Earning  AR  points  became  the  primary  goal  at  a 
number  of  school  sites. 

Staff  Development  with  regard  to  Computer  Technology 

It  is  important  to  note  that  the  district  offered  staff  development  classes  coordinated  by  the  district’ s 
Reading  Renaissance  coordinator,  some  of  which  were  labeled  “Computer  Classes”.  Participants  in  these  classes 
could  receive  credit  towards  salary  increments.  The  “Computer  Classes”  focused  on  helping  teachers  understand 
how  the  STAR  Reading  Assessment  program  worked,  how  to  manage  the  Accelerated  Reader  software,  how  to 
access  the  various  reports  in  the  software,  and  how  to  use  the  Book  Guide  components  of  the  software.  The  other 
classes  focused  on  goal  setting  for  AR  use  in  one’s  classroom;  assessing  the  status  of  the  students’  in  one’s 
classroom  and  maintenance  of  student  reading  logs;  diagnosis  and  intervention  to  assist  students;  how  to  match 
books  to  students;  adjusting  reading  levels;  model  mini-lessons  for  classroom  use;  and  understanding  the 
requirements  to  seek  “Model”  and  “Master”  Reading  Renaissance  certification.  These  classes  were  offered  in  a 
central  location  or  at  Hoyt  Elementary  which  had  served  as  the  initial  pilot  site  for  AR. 

Classroom  educators  who  received  Reading  Renaissance  certification  proudly  displayed  their  achievements 
on  their  classroom  door.  When  all  classroom  educators  received  a certain  level  of  certification,  the  school  would 
also  receive  the  related  Model  or  Master  Renaissance  (http://www.renlearn.com/rencertification/default.htm, 
Accessed  March  20,  2003)  school  label.  Ely  Elementary  was  reported  as  the  first  school  in  Arizona  to  earn  the 
Master  Renaissance  school  status.  In  2001-02,  Hoyt,  Rian,  and  Sloan  Elementary  schools  all  reported  achieving 
Model  or  Master  Renaissance  school  status.  This  achievement  was  reported  to  the  Arizona  Department  of  Education 
by  the  school  principals  in  the  annual  school  report  card  as  one  of  the  three  or  four  school  level  accomplishments  for 
the  year.  Pule  Elementary  reported  the  fact  that  all  classroom  teachers  at  the  school  had  received  training  in  the 
Reading  Renaissance  program  in  2002-03  as  an  accomplishment.  These  highlight  the  importance  placed  by 
administrators  and  teachers  on  the  school- wide  implementation  of  the  Reading  Renaissance  program. 

Use  of  Computer  Lab  Sessions 

Students  typically  used  the  school  computer  lab  merely  for  a total  of  15  hours  or  1%  of  1,035  instructional 
hours  in  a school  year.  Access  to  school  use  of  computers  for  the  typical  student  was  insignificant  in  relation  to  the 
amount  of  instruction  time  at  school.  Whereas,  a large  number  (87%)  of  the  students  in  participant  classrooms  said 
that  they  had  access  to  a home  computer.  Use  of  the  scheduled  computer  lab  sessions  were  dedicated  to  cultivation 
of  technical  skills  (e.g.,  keyboarding  and  learning  the  use  of  word  processing,  spreadsheet  software,  Internet).  The 
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district’s  technology  integration  model  focused  on  “vocational  life  skills”  as  evidenced  by  the  enacted  key-boarding 
curriculum  and  the  technology  education  curriculum.  The  mission  for  integration  of  computers  into  the  district  was 
of  the  view,  “unless  we  have  technology  in  our  schools,  kids  won't  be  able  to  get  jobs  in  an  increasingly 
computerized  world.” 

Computer  lab  instruction  was  supported  by  part  time  (50%)  district  assigned  computer  resource  technology 
specialists  who  were  also  certified  teachers.  These  specialists  typically  served  two  different  campuses,  part-time  on 
each  campus.  Their  role  was  to  support  instruction  in  the  computer  lab  by  designing  and  introducing  lessons.  The 
classroom  teacher  was  expected  to  complete  the  project  started  by  the  specialist.  However,  teachers  used  the 
computer  lab  sessions  as  specials  and  did  not  stay  in  the  lab  during  instruction.  Teachers  were  not  evaluated  on  their 
use  of  computers  in  education  and  did  not  feel  responsible  for  instruction  in  the  lab.  The  district’s  project-based 
technology  integration  plans  functioned  in  isolation  from  classroom  instruction.  Additionally  these  lab  sessions  did 
not  challenge  students’  prior  experience  with  computers  at  home. 

In  2001-03,  the  average  per  pupil  expenditure  for  participant  schools  was  $3,323  and  comparable  to  the 
state  average  of  $3,012.  The  schools  were  well  funded  by  state  standards  and  the  district  primarily  served  children 
from  largely  middle  class  families.  These  families  typically  had  middle  class  aspirations  for  their  children  and 
expected  school  to  provide  their  children  a “good”  education.  However,  access  to  computer  technology  for  students 
in  school  was  very  limited.  Chiefly,  students’  ability  to  think  for  themselves,  think  critically,  and  appropriate 
computer  technologies  as  learning  tools  were  not  facilitated  and  exploited  by  school. 

Findings 

An  interpretive  comparison  (Erickson,  1986)  of  the  patterns  in  the  set  of  eight  case  studies  led  to  the 
following  assertions: 

1.  Classroom  use  of  computers  was  defined  by  the  use  of  the  Accelerated  Reader  program  and  its  uncritical 
acceptance  as  a software  tool  to  manage  students’  reading  practices. 

2.  School  use  of  computer  technology  was  similar,  ordinary,  and  monotonous. 

3.  Access  to  school  use  of  computers  for  the  typical  student  was  insignificant  in  relation  to  the  amount  of 
instruction  time  at  school. 

4.  Use  of  computer  technology  in  education  was  symbolic  and  did  not  represent  curricular  integration  with 
technology. 

The  findings  suggest  that  schools  need  to  establish  clear  academic  goals,  determine  how  the  use  of  computer 
technology  will  help  achieve  these  school  goals,  and  employ  new  models  of  professional  development  that  go 
beyond  the  training  paradigm. 

The  technology  infusion  model  initially  adopted  by  the  district  in  1994  was  organized  by  a committee 
composed  of  administrators,  teachers,  and  parents.  The  committee  conducted  parent  and  teacher  surveys  to  seek 
input  on  the  district’s  “needs  relative  to  technology  in  the  classroom.”  Yet,  in  a rapidly  growing  district  such  as 
Awan  Public  Schools,  seven  years  later,  in  2001,  the  original  model  along  with  the  technology  education  standards 
and  technology  skills  performance  goals  for  learners  seemed  out  of  place.  The  1993-1994  technology  committee  that 
developed  plans  for  technology  infusion  into  Awan’s  public  schools  stated  its  mission  as,  “to  maximize  the 
resources  available  to  educate  students  for  life-long  learning  of  vocational  and  life  skills  through  the  utilization  of 
modern  technology.”  This  mission  as  implemented  by  the  district’s  administration  and  staff  gave  the  impression  of 
modernity  whereas  the  observed  use  of  computers  in  education  was  rather  pedestrian. 

The  school  district  had  grown  rapidly.  In  a span  of  five  years,  from  1997-98  to  2001-02,  enrollment  had 
increased  by  45%.  At  the  same  time  hardware  and  software  technology  had  advanced  considerably  and  ideas  about 
the  place  of  computers  in  education  were  still  unresolved.  The  initial  model  focused  on  the  cultivation  of  “vocational 
life  skills”  as  evidenced  by  the  key-boarding  curriculum  and  the  technology  education  curriculum.  The  mission  for 
the  Awan  Public  Schools,  with  regard  to  computers,  seemed  to  be  simple:  Unless  we  have  technology  in  our 
schools,  our  kids  won't  be  able  to  get  jobs  in  an  increasingly  computerized  world.  Consequently,  the  focus  on 
preparing  students  for  vocational  and  life-skills  justified  the  investment  in  school  computer  technology. 

In  2002-03  was  this  goal  still  valid  for  a population  that  was  increasingly  upwardly  mobile  with  44%  of 
Awan’s  labor  force  in  professional  occupations?  Or  for  that  matter,  is  a curriculum  of  the  kind  adopted  and 
implemented  by  Awan  Public  Schools — a curriculum  that  aims  to  “teach”  students  about  commonplace  software 
technology-appropriate  for  student  populations  of  any  socio-economic  level?  In  2000,  Larry  Cuban,  a critic  of 
techno-enthusiasts,  said: 

We  worry  about  teaching  keyboarding  today  but  it  won't  be  needed  when  voice  activation  becomes 

common  in  a few  years.  The  dogma  will  only  be  replaced  when  people  realize  how  quickly  it  is  all 
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changing.  Kids  don't  need  years  of  computer  exposure  to  succeed.  People  with  no  computer  background 
generally  catch  on  in  a few  weeks-a  few  months  tops.  (Salpeter,  2000) 

Then  what  is  computer’s  place  as  a “modern  technology”  or  “new  technology”  in  public  education?  The  population 
served  by  Awan  Public  Schools  was  largely  middle  class  and  upwardly  mobile  as  noted  by  students’  reports  of 
parents’  occupations  and  confirmed  by  data  from  the  2000  census.  In  July  2003,  the  town  of  Awan  was  designated 
as  the  fastest  growing  city  of  242  cities  with  populations  of  100,000  or  more  in  the  nation.  Between  April  1,  2000, 
and  July  1,  2002,  Awan’s  population  had  grown  by  nearly  23%.  According  to  the  2000  census,  the  racial  diversity  in 
the  town  of  Awan  was  as  follows:  80%  White,  12%  Hispanic  or  Latino,  4%  Asian  American,  3%  African  American, 
and  1%  Native  American.  Further,  according  to  the  2000  census,  Awan  had  close  to  35,000  households.  Only  8%  of 
these  were  female  led  households  with  children  but  no  adult  male  present,  whereas,  82%  of  the  households  were 
families  with  children.  In  2000,  44%  of  the  town’s  labor  force  was  employed  in  management  or  related  professional 
occupations,  31%  in  sales  and  office  occupations,  10%  in  service  occupations,  6%  in  construction,  extraction,  and 
maintenance  occupations,  and  8%  in  production,  transportation,  and  materials  moving  occupations.  The  district 
primarily  served  children  from  largely  middle  class  families.  These  families  typically  had  middle  class  aspirations 
for  their  children  and  expected  school  to  provide  their  children  a “good”  education. 

Cuban  (2001,  p.  156-159)  argued  that  the  acquisition  of  computers  by  public  schools  is  symbolic  of  high 
status  and  modernity: 

By  the  late  1990s,  the  computer — like  past  mechanical  marvels  such  as  the  steam  engine,  the  railroad 
locomotive,  and  the  airplane — had  become,  among  other  things,  a high-status  symbol  of  power  and 
modernity... Woe  to  the  school  leader  unable  to  show  patrons  and  visitors  rooms  full  of  machines.  A “good” 
school  has  become,  by  definition,  a technologically  equipped  one. 

Awan  Public  Schools  certainly  fit  the  description  of  a “good”  school.  Each  school  showcased  a computer  lab  and 
each  classroom  contained  one  or  two  Internet  accessible  computers.  Nevertheless,  the  school  and  classroom 
computers  were  largely  used  to  emphasize  technical  skills  and  technology  education.  The  implementation  of  the 
Accelerated  Reader  and  Math  programs  suggested  teacher  efficiency  and  productivity.  In  theory  these  programs 
freed  up  teachers’  from  manually  creating,  administering,  and  grading,  individualized  reading  and  math  assessments. 
The  Accelerated  Math  software  program  used  Scantron — bubble  sheet  technology  to  assess  student  practice  and 
assessments.  The  software  program  was  chiefly  used  to  print  additional  practice  assignments  for  students  to  practice 
math  skills  as  a supplement  to  classroom  instruction.  These  instances  of  classroom  use  of  computer  technology  were 
also  geared  to  increasing  student  productivity  by  providing  additional  practice  and  immediate  feedback. 

The  observed  applications  of  computer  technology,  particularly  the  Accelerated  Reader  and  Math  programs 
were  believed  to  raise  student  achievement,  promote  better  teaching  as  their  use  provided  the  elementary  grade 
teacher  with  more  time  to  provide  individualized  instruction,  and  demonstrate  school’s  use  of  technology  to 
document  students’  academic  progress  in  reading  and  mathematics.  Yet,  there  was  no  research-based  evidence  that 
these  programs  were  helpful  in  increasing  student  achievement.  Many  scholars  have  noted  that  American  culture  has 
long  valued  “modernity,”  as  meaning  that  schools,  like  businesses,  must  appear  efficient,  productive,  business-like, 
pioneering,  and  futuristic  (Cuban,  2001;  Spring,  1998). 

Conclusion 

While  giving  a semblance  of  modernity,  essentially,  the  observed  use  of  classroom  computers  in  eight 
Arizona  upper  elementary  as  defined  by  the  Accelerated  Reader  program  chiefly  served  the  amplification  purposes 
of  technology.  It  represented  the  use  of  computers  to  improve  teacher  and  student  efficiency.  The  observed  practices 
of  technology  integration  did  not  represent  curricular  integration  of  technology;  instead  they  served  to  draw  attention 
to  the  isolation  of  classroom  instruction  from  computer  use.  Computer  technology  use  was  largely  symbolic  of 
modernism  and  only  a token  representative  of  the  notion  of  curricular  technology  integration. 

The  district’ s technology  support  structure  was  designed  to  be  efficient  from  the  viewpoint  of  maintaining 
the  technology.  But  this  structure  prevented  teachers  from  attempting  innovation  with  the  available  computer 
technology.  The  mandatory  procedures  to  install  software  programs  that  were  not  already  available  on  the  school 
computers  were  cumbersome  and  time  consuming.  In  addition,  the  age-graded  school  with  self-contained 
classrooms,  and  a district  curriculum  that  required  teachers  to  teach  subjects  in  allotted  time  periods,  forced  teachers 
to  “ration”  their  energy  and  time  (Cuban,  1993).  The  structure  of  schooling  in  participant  classrooms  was  largely 
designed  for  whole-group  instruction.  Explicit  classroom  rules  prevented  students  from  working  in  groups  and 
sharing  their  learning  with  each  other  and  their  teacher.  Teachers  had  to  maintain  classroom  order,  transmit 
knowledge,  and  determine  whether  students  learned  the  subject  matter.  These  tasks  were  to  be  done  efficiently 
within  the  pre- specified  instruction  time  of  a school-day.  Thus  teachers  engaged  in  instructional  techniques  that 
were  compatible  with  the  district’s  and  the  school’s  existing  structures. 
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Very  few  teachers  in  the  study  were  willing  to  explore  a student-centered  approach  that  would  force  them 
to  let  go  of  set  class  periods,  allocated  instruction  time  for  subject  matter,  and  classroom  routines.  Technology  can 
play  a significant  role  in  supporting  a student-centered  approach  to  curriculum.  Such  an  approach  would  associate 
knowledge  construction  as  a strategy  that  enhances  learning;  helps  personalize  concepts;  gain  deeper  understanding; 
expand  thinking  and  problem-solving  skills;  and  generate,  develop,  explore,  model,  and  synthesize  ideas.  In  direct 
contrast,  the  organizational  structures  of  the  district  and  the  school  encouraged  regularities  in  instruction  that  were 
ordinary  and  monotonous.  These  structures  did  not  promote  innovation  in  the  instructional  use  of  technology. 

School  reform  with  technology  would  require  that  teachers  completely  refurbish  their  fundamental  methods  of 
classroom  operation  and  few  teachers  demonstrated  the  will  to  disturb  the  stability  of  a controlled  and  orderly 
classroom. 

Integrating  computer  technology  into  instruction  is  complex.  Merely  providing  an  Internet  accessible 
computer  in  every  classroom  and  scheduling  each  class  to  use  the  school  computer  lab  once  every  week  does  not 
meet  the  requirements  of  technology  integration.  Students  in  this  study  had  limited  access  to  computer  technology  in 
school.  Expecting  such  limited  access  to  technology  to  significantly  change  the  process  of  learning  would  be 
unreasonable.  Each  of  the  teachers  and  administrators  in  the  study  noted  the  difficulty  in  having  students  use  the 
only  computer  in  the  classroom  and  the  problem  of  the  availability  of  a computer  lab  session  once  every  six  school 
days.  Despite  reports  of  increasing  access  to  computers  in  both  school  and  home,  the  reality  of  the  public  school 
classroom  may  be  that  only  limited  access  to  computers  is  the  norm. 

Having  one  or  two  computers  in  the  classroom  does  not  constitute  adequate  access  to  technology  and  it 
does  not  lead  to  significant  student  use.  Computer  technologies  cannot  have  a significant  impact  on  student  learning 
when  teachers  and  students  do  not  have  the  opportunity  to  access  and  use  the  technologies  on  a regular  basis.  All  the 
same,  the  most  noticeable  and  commonplace  use  of  computers  in  participants’  classrooms  and  schools  were 
restricted  to  the  use  of  computer-based  reading  comprehension  quizzes.  In  addition,  use  of  the  Accelerated  Reader 
program  was  representative  of  schools’  need  to  use  computer  technology  or  any  program  that  seemed  to  enhance 
student  achievement  as  measured  by  standardized  tests.  It  was  indicative  of  the  larger  school  accountability  model 
under  which  schools  operate  today.  Policy  makers  hold  schools  accountable  through  use  of  tests  of  student 
achievement  that  can  be  widely  and  uniformly  administered  across  schools.  Although  there  have  been  advances  in 
measurement  of  student  achievement  using  standardized  tests,  they  are  not  compatible  with  our  increasing 
understanding  of  learning  and  the  development  of  expertise.  A major  research  emphasis  is  necessary  on  measures  of 
student  achievement  that  reflect  the  characteristics  of  learning  advocated  by  the  National  Research  Council  (2000). 
What  researchers  consider  are  exemplary  uses  of  computer  technology  to  enhance  learning  and  what  educational 
policy  makers  demand  that  schools  measure  regarding  student  achievement  are  mismatched.  Given  the  prevalent 
school  accountability  disposition  in  the  nation,  schools  cannot  be  expected  to  seriously  engage  with  new 
technologies  to  enhance  learning  in  desirable  ways. 
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Designing  an  Interdisciplinary  Degree  for  Performance  Improvement: 

A Case  Study  at  the  University  of  Georgia 

Elinor  Greene 
The  University  of  Georgia 

Abstract 

In  August  2002,  the  University  of  Georgia' s College  of  Education  started  the  Instructional  Psychology, 
Training  and  Technology  (IPTT)  bachelor's  degree,  an  interdisciplinary  degree  that  integrates  Instructional  Design 
and  Technology,  Educational  Psychology,  Adult  Education,  Occupational  Studies,  and  Workforce  Development. 

This  degree  was  the  first  of  its  kind,  because  it  provided  these  critical  skills  and  competencies  at  the  bachelor's  level 
to  adults  already  working  in  the  field.  The  challenges  and  rewards  in  providing  an  interdisciplinary  bachelor's 
degree  to  non-traditional  students  in  an  off-campus,  metropolitan  setting  are  discussed,  as  well  as  the  reasons  why 
the  degree  was  abruptly  discontinued  in  the  midst  of  rapid  growth  and  growing  reputation.  This  degree  could  serve 
as  a model  program  for  institutions  serving  non-traditional  students  in  a metropolitan  environment. 

Increasingly,  businesses  have  become  a larger  provider  of  educational  services  for  adults  than  higher 
education.  Government  agencies,  hospitals,  community  agencies,  unions,  and  a host  of  other  private  sector,  federal, 
state,  local,  and  voluntary  associations  are  providing  educational  services  to  their  employees,  customers,  and 
suppliers.  Rapid  technological  change,  globalization,  restructured  workplaces,  labor  market  shifts,  and  pressure  for 
reduced  costs  and  increased  productivity  have  increased  the  demand  for  professionals  capable  of  developing  human 
capital  through  the  application  of  several  disciplines.  People  working  in  areas  of  developing  employees,  a 
workforce,  or  an  organization,  with  the  goal  of  producing  competent,  confident,  and  satisfied  personnel;  without 
formal  or  with  limited  education;  become  successful  through  a process  of  trail  and  error,  coupled  with  the  varied 
experiences  received  through  attending  conferences,  seminars,  or  workshops.  Moreover,  formal  education  for 
skilled  personnel  in  the  areas  of  adult  education,  instructional  design  and  technology,  career  development,  change 
management,  and  the  science  of  learning  and  motivation  has  largely  been  limited  to  the  graduate  level,  and  is 
generally  limited  to  one  or  two  programs  or  departments. 

People  working  in  the  broad  fields  of  training,  instructional  design,  and  human  resource  development  often 
entered  their  craft  through  various  avenues  and  developed  their  own  career  paths,  or  stumbled  into  their  careers, 
without  ever  completing  their  bachelor’s  degree.  For  individuals  who  received  their  training  on  the  job,  standards 
have  been  created  by  professional  organizations  and  accrediting  bodies  to  ensure  a minimum  degree  of  competency; 
for  example,  the  Association  for  Educational  Communications  and  Technology  (AECT),  the  International  Society 
for  Performance  Improvement  (ISPI),  and  the  International  Board  of  Standards  for  Training,  Performance  and 
Instruction  (IBSTI). 


UGA  Expands  its  Reach  into  the  Community 

In  1999,  leaders  in  Gwinnett  County  requested  that  the  University  of  Georgia  begin  providing  bachelor’s 
degrees  in  Gwinnett  County,  a rapidly  growing  county  in  the  metropolitan  Atlanta  area.  The  Board  of  Regents 
approved  this  request  and  UGA  committed  to  providing  three  bachelor’s  degrees  in  2002.  UGA  had  already  been 
serving  the  greater  Atlanta  area  by  providing  evening  Master’s  degrees  in  Gwinnett  County  for  several  years,  with 
great  success.  In  addition,  Georgia  Perimeter  College  had  been  providing  Associates  degrees  at  the  same  location. 
In  fall,  2002,  UGA  began  offering  three  separate  bachelors  degrees,  (1)  Instructional  Psychology,  Training,  and 
Technology,  (2)  Business  Administration,  and  (3)  Biological  Sciences.  By  2004,  UGA  was  offering  seven 
bachelor’ s degrees  in  Gwinnett. 


Interdisciplinary  Learning 

In  the  field  of  human  performance  technology,  there  are  many  ways  to  affect  change;  all  beginning  with  a 
thorough  needs  assessment  and  requiring  a myriad  of  blended  solutions,  resulting  in  summative  evaluation  which 
can  be  equated  with  return  on  investment  for  companies  seeking  a productive  and  competitive  workforce  and 
monetary  returns  for  their  shareholders.  The  interventions  for  improving  performance  and  managing  change  to 
ensure  systemic  benefits  require  a broader  perspective  and  a greater  range  of  disciplines  from  which  to  draw  to 
accurately  analyze  and  solve  complex  business  problems,  without  harming  those  processes  that  are  working  well 
within  and  outside  the  organization.  Interdisciplinary  programs  provide  an  opportunity  for  students  with  career 


224 


goals  that  can  best  be  met  by  drawing  from  the  knowledge  base,  methods,  and  experiences  of  more  than  one 
discipline.  Figure  1 depicts  a model  of  human  performance  technology  that  requires  an  interdisciplinary  perspective 
for  analysis  of  needs  and  interventions. 


Figure  1.  One  Model  of  Human  Performance  Technology  (Adapted  from  ISPFs  HPT  Model,  2004) 

A Model  of  Performance  Technology 


Performance  Analysis 


Cause  Analysis 


Intervention 
Selection  and  Design 


Promoting  learning  and  change  in  work  processes  beyond  a single  discipline  is  a primary  goal  toward  a 
construction  that  integrates  and  synthesizes  multiple  perspectives.  Schommer  (1994)  noted  that  interdisciplinary 
learning  shifts  the  focus  from  memorizing  facts  in  a single  theme  to  the  interpretation  and  application  of  knowledge 
across  several  contexts.  This  was  a wake  up  call  for  institutions  of  higher  education  to  develop  programs  of  study 
that  enhance  students’  critical  abilities  to  solve  complex  business  problems. 

Michael  & Balraj  (2003)  reported  that  few  institutions  have  established  procedures  for  developing, 
regulating,  and  evaluating  interdisciplinary  degree  programs.  A report  by  the  Carnegie  Foundation  (1998)  noted  the 
lack  of  interdisciplinary  bachelor’ s degrees  at  research  universities  and  cited  such  barriers  as  traditionally  defined 
departments  and  lack  of  sponsorship.  Interdisciplinary  programs  require  support  by  an  administration  that 
demonstrates  commitment,  including  monitory  support,  and  recognition  of  the  faculty  members  willing  to  invest 
themselves  and  commit  to  the  quality  of  interdisciplinary  education  (Einhellig  & Marre  1999).  These  issues  are 
obvious  to  Research  1 institutions  where  research  and  external  funding  are  the  primary  focus. 

Einhellig  & Marre  (1999)  suggested  that  an  interdisciplinary  curriculum  might  emerge  as  a new  and 
distinct  field  of  studies.  A noteworthy  example  is  UGA’s  Human  Resource  - Organizational  Development  (HROD) 
Master’s  degree,  also  offered  in  Gwinnett  County.  The  College  of  Education’s  Adult  Education  and  Occupational 
Studies  departments  joined  forces  to  develop  this  unique  degree,  which  incorporates  perspectives  and  faculty  from 
both  disciplines.  IPTT’s  interdisciplinary  curriculum  would  serve  as  an  ideal  model  for  institutions  seeking  to 
establish  a distinct  field  of  study  to  meet  the  needs  of  both  students  and  organizations. 

Program  Development 

In  2001,  a market  survey  conducted  by  businesses  in  Gwinnett  County  revealed  a need  for  an 
undergraduate  degree  that  would  provide  graduates  with  the  skills  and  competencies  necessary  to  help  its  growing 
number  of  public  and  private  organizations  become  more  productive  and  successful.  Gwinnett  County  is  home  to 
more  than  930  manufacturing  companies,  over  981  high-tech  companies,  and  235  international  companies  (Gwinnett 
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Chamber  of  Commerce,  2002).  Gwinnett  County  has  almost  doubled  its  population  in  each  of  the  past  three 
decades,  and  it  is  predicted  that  its  population  will  be  over  1,000,000  by  2020.  Moreover,  Gwinnett  County  is 
diverse.  Students  attending  Gwinnett  County’s  public  schools  represent  language  and  cultural  backgrounds  from  an 
estimated  115  countries,  which  range  alphabetically  from  Afghanistan  to  Zimbabwe  (Morgan,  2002). 

The  Interdisciplinary  IPTT  degree  was  developed  by  the  University  of  Georgia’ s College  of  Education  in 
response  to  a formal  request  by  businesses  in  Gwinnett  County,  just  north  of  Atlanta.  The  curriculum  was  designed 
to  enhance  the  skills  and  knowledge  of  adults  in  the  metropolitan  Atlanta  area  who  have  work  experience  in  the 
areas  of  training,  organizational  development,  human  resource  development,  and  other  areas  of  performance 
improvement  or  who  were  seeking  a career  change,  and  who  never  completed  their  bachelor’s  degree,  and  are  often 
penalized  by  losing  their  jobs  or  being  overlooked  for  promotion. 

Based  on  a formal  request  from  the  community,  there  was  no  question  that  the  first  bachelor’s  degree 
offered  by  UGA’s  College  of  Education  would  require  an  interdisciplinary  focus  geared  to  meet  the  growing 
performance  needs  of  working  adults.  Therefore,  in  2002,  four  departments  in  UGA’s  College  of  Education  created 
and  began  offering  a unique,  interdisciplinary  undergraduate  major  as  a collaborative  effort.  At  the  time,  these  four 
departments  were  Educational  Psychology,  Adult  Education,  Occupational  Studies,  and  Instructional  Technology. 
Since  that  time,  the  College  of  Education  reorganized,  which  affected  representation  on  the  IPTT  advisory 
committee,  but  did  not  affect  the  curriculum  in  any  major  way.  The  advisory  committee,  which  previously  consisted 
of  department  heads  of  the  four  departments,  became  the  department  heads  of  three  departments,  namely, 
Educational  Psychology  - Instructional  Technology  (EPIT),  Occupational  Studies,  and  Adult  Education. 

The  author  started  the  IPTT  program  at  the  Gwinnett  Campus  in  2002  and  served  as  program  director, 
recruiter,  advisor,  liaison  between  Gwinnett  and  the  main  campus,  internship  supervisor,  and  instructor  of  five  major 
courses  in  the  degree.  She  left  the  program  in  July  2005,  after  the  decision  was  made  to  discontinue  undergraduate 
degrees  in  Gwinnett. 


IPTT  Program  Goals 

The  interdisciplinary  Instructional  Psychology,  Training,  and  Technology  bachelor’s  degree  completion 
program  was  designed  to  provide  students  with  the  skills,  competencies,  and  experience  to  help  organizations  in  the 
greater  Atlanta  area  identify  their  performance  needs  and  to  provide  measurable  improvements  through  training  and 
non-training  interventions. 


Major  Coursework 

As  most  of  the  IPTT  students  were  non-traditional  and  working  in  the  greater  Atlanta  area,  all  courses  meet  in 
the  evenings,  once  per  week,  with  two  courses  scheduled  back  to  back  each  evening.  Some  courses  are  offered  on  line, 
or  through  blended  delivery.  Table  1 provides  a complete  listing  of  major  coursework. 


Table  1.  Instructional  Psychology,  Training  and  Technolo^ 

*y  Major  Coursework 

Instructional  Technology 

Adult  Education 

Instructional  Design 

Design  & Development  Tools  (web  development) 
Project  Management 

Learning  Environment  Design  (web  based  learning 
environments) 

Adult  Learning  in  the  Workplace 
Adult  Development  in  the  Workplace 
Organizational  Learning  in  the  Workplace 
Diversity  & Globalization  of  the  Workplace 

Educational  Psychology 

Occupational  Studies 

Cognitive  Foundations  for  Education 
Motivational  Foundations  for  Education 
Research  Methods 
Applied  Educational  Assessment 

Needs  Assessment  for  Workforce  Development 
Career  Development  in  the  Workplace 
Situated  Learning  in  the  Workplace 
Internship  in  Business  or  Industry 

Demographics,  Student  Involvement,  and  Faculty 

All  IPTT  majors  have  been  working  from  five  to  twenty  years  and  they  range  in  age  from  23-55,  with  an 
average  age  of  34.  Since  its  inception,  approximately  41  students  have  graduated  and  the  remainder  of  the  students 
will  graduate  by  2007. 

In  2004,  IPTT  students  formed  the  Human  Performance  Technology  Student  Association  (HPTSA),  a 
chartered  UGA  organization,  and  they  met  on  a regular  basis,  networking  and  bring  in  speakers  of  interest  to  our 
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students.  Even  our  graduates  continued  to  attend  the  monthly  HPTSA  meetings  and  fundraisers,  and  they  still 
participate  in  events. 

Most  students  have  full  time  jobs,  working  in  a variety  of  industries  in  the  greater  Atlanta  area.  They 
pursued  internships  in  career  centers,  multimedia  development  companies,  non-profit,  and  for  profit  industries.  We 
also  started  an  internship  program  with  the  Gwinnett  Detention  Center  to  help  inmates  with  reentry  skills  to  reduce 
the  revolving  door. 

Our  instructors  are  tenure  track  faculty  from  the  four  respective  departments  and  programs,  who  typically 
travel  from  Athens  once  a week  to  teach  their  classes.  They  generally  teach  graduate  students  in  Athens,  which  adds 
richness  to  the  learning  experience. 


Number  of  Majors  Enrolled 

In  our  first  semester,  we  started  with  17  students,  a major  accomplishment  for  a newly  designed  and 
unknown  degree.  This  was  largely  due  to  a marketing  blitz  on  National  Public  Radio.  Within  a month,  drastic 
budget  cuts  eliminated  our  ability  to  advertise,  which  adversely  affected  our  growth.  The  degree  maintained  a 
steady  growth  since  inception,  largely  through  word  of  mouth,  and  we  had  56  students  enrolled  at  its  peak.  Figure  2 
depicts  growth  and  projected  enrollment  of  majors  in  the  IPTT  degree,  which  will  rapidly  decline  as  the  program  is 
phased  out. 

Figure  2.  IPTT  Growth  and  Projected  Enrollment 
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Future  of  the  Program 

In  2005,  Gwinnett  County  made  the  decision  to  start  its  own  four-year  college,  and  the  board  of  regents 
agreed  through  a unanimous  vote.  The  Georgia  General  Assembly  approved  the  new  college.  As  a result, 
University  of  Georgia  Undergraduate  Programs  are  no  longer  admitting  new  students,  as  the  new  college  will  be 
housed  in  the  same  facility  that  UGA  and  Georgia  Perimeter  College  currently  share.  Both  UGA  and  Georgia 
Perimeter  College  will  cease  teaching  undergraduate  courses  in  Gwinnett  County  in  2008.  Because  the  Instructional 
Psychology,  Training,  and  Technology  degree  was  created  for  a metropolitan  campus,  there  will  be  no  comparable 
undergraduate  degrees  offered  in  Athens,  UGA’s  main  campus. 

Summary 

UGA’s  interdisciplinary  IPTT  bachelor's  degree  was  created  in  response  to  a formal  request  by  business 
owners  in  Gwinnett  County.  The  degree  was  designed  for  non-traditional  working  students  and  it  only  existed  in 
Gwinnett.  Clearly,  this  unexpected  decision  by  Gwinnett  County  and  the  Board  of  Regents  will  adversely  affect  the 
futures  of  the  students,  staff,  and  faculty  who  were  involved  in  growing  and  promoting  this  unique  UGA  bachelor’s 
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degree  in  Gwinnett.  However,  the  model  would  work  well  in  an  institution  serving  non-traditional  students  in  a 
metropolitan  area. 
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Abstract 

Designing  online  courses  should  be  aimed  at  broadening  students'  knowledge,  improving  their  makings, 
and  developing  their  comprehensive  abilities.  It  should  be  guaranteed  to  achieve  the  effectiveness  by  means  of 
appropriate  teaching  strategies.  This  paper  consists  of  four  parts,  explaining  the  fundamental  requirements  of 
English  writing  teaching  in  web  settings,  and  discusses  the  designing  of  its  teaching  contents  and  strategies.  Some 
beneficial  suggestions  are  put  forwarded,  based  on  the  authors'  practical  online  teaching  experiences. 

An  overview  of  online  teaching  in  the  world  enables  us  to  see  its  realistic  and  potential  advantages  more 
clearly,  and  online  courses  have  aroused  people’s  increasing  attention  in  China,  including  various  online  foreign 
language  teaching  courses.  Although  these  courses  have  their  own  traits,  they  are  by  and  large  in  the  infant  stage. 
With  increasing  perfection  of  web  technology  and  people’s  recognition  of  online  foreign  language  teaching,  we 
believe  that  the  prospect  of  foreign  language  teaching  in  online  settings  is  promising.  Based  on  this  awareness,  we 
combine  design  of  the  online  English  writing  course  and  teaching  practice  into  an  integral  research  project  so  that 
this  teaching  mode  goes  on  in  a scientific,  systematic  and  standardized  status  from  the  very  beginning. 

Basic  requirements  on  designing  the  online  English  writing  course 

Is  the  online  writing  course  a digitalized  and  networked  copy  of  the  conventional  writing  curriculum?  Is  it 
an  online  VOD  of  the  classroom  teaching?  What  should  design  of  an  online  writing  course  include?  And  What 
strategy  should  be  adopted  in  teaching  the  course?  These  are  the  questions  that  should  be  answered  in  designing  a 
good  online  foreign  language  writing  course. 

Online  teaching  not  only  breaks  through  the  conventional  classroom  teaching  mode  but  breaks  through  the 
limitations  of  ineffective  communication  and  idea  exchange  in  conventional  distance  education  [1].  Online  teaching 
wants  to  establish  a novel  teaching  mode,  which  is  far  from  simply  placing  the  teaching  content  on  the  web  and 
letting  the  student  read  it  passively;  what  it  needs  more  includes: 

1)  Enabling  the  teacher  and  the  student  to  communicate  and  exchange  ideas  freely  so  that  the  student  feels 
that  he  or  she  is  learning  under  the  instruction  of  the  teacher  rather  from  the  computer.  In  other  words,  teaching  and 
learning  is  being  conducted  in  a human-based  manner. 

2)  Enabling  the  teacher  to  organize  the  student’s  learning  activities,  tempt  the  student  to  learn  actively,  and 
help  the  student  solve  difficulties  in  his  or  her  studies. 

3)  A good  function  of  evaluating  and  monitoring  teaching  feedback  of  the  online  course  so  that  the  teacher 
is  able  to  know  characteristics  and  learning  progress  of  the  learner  as  the  principal  part  of  the  teaching  activity  and 
to  offer  individualized  assistance  and  improve  the  outcome. 

4)  Enabling  the  core  link  in  the  conventional  teaching  mode  to  be  well  implemented  on  line  with  respect  to 
lecturing,  learning,  taking  examinations,  giving  assignments,  discussing,  evaluating  and  answering  questions. 

5)  implementing  quality  education,  transforming  conventional  knowledge  teaching  into  ability  teaching, 
where  the  online  course  should  be  able  to  support  a variety  of  teaching  modes,  especially  the  question-based 
explorative  and  coordinative  learning  mode  in  the  form  of  writing  forums,  internal  E-mail  and  English  chatting 
rooms  as  interactive  teaching  zones [2]. 

Course  development  and  design  are  of  crucial  importance  in  online  teaching  settings  including  the  English 
writing  course,  the  purposes  of  which  are  to  motivate  initiative  of  the  teacher  and  the  student,  broaden  the  curricular 
scope,  and  improve  the  teaching  outcome  and  quality.  At  the  same  time,  the  designer  should  condense  abundant 
related  information  into  the  online  course  so  that  interaction  between  the  student  and  the  teacher  can  be  developed 
on  a more  informative  and  colorful  level,  and  the  student’ s interest  in  writing  can  be  activated  by  taking  advantage 
of  interaction  of  the  online  course.  The  students  should  be  encouraged  to  tap  more  materials  and  think  more 
creatively. 
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The  art  of  designing  teaching  contents  of  the  online  English  writing  course 

Essay  writing  is  a complex  psychological  process  on  the  part  of  the  student,  involving  the  use  of 
comprehensive  knowledge  and  abilities.  Online  writing  is  intended  to  optimize  teaching  and  training  in  three 
aspects:  emotion,  knowledge  and  ability,  so  as  to  upgrade  the  student  with  respect  to  aspiration,  quality,  writing 
knowledge  and  ability.  For  this  purpose,  design  of  the  teaching  content  is  mainly  concerned  with  topic  making,  topic 
discussion  and  practice,  literature  support  and  master  page  of  teaching. 

Topic  making 

To  ensure  stability  of  the  curriculum,  the  teacher  is  usually  quite  prudent  in  selecting  sensitive  topics  and 
model  writings.  To  make  full  use  of  Internet’s  dynamic  interaction,  online  writing  teaching  is  not  restrained  by  such 
limitations;  on  the  contrary,  it  usually  selects  topics  of  high  timeliness  and  literary  works  of  high  sensation  so  far  the 
language  is  standardized,  such  as  a paragraph  from  a newspaper  column,  or  a broadcast  and  TV  program.  Of  course, 
it  is  necessary  to  deal  well  with  the  relationship  between  material  selection  and  language  ability  training;  the  former 
should  be  subordinated  to  the  latter.  To  make  full  use  of  the  web  advantage  in  language  resources,  we  combine  the 
“blueprint”  with  the  “resource”.  For  example,  we  not  only  provide  the  student  with  materials  about  various  writing 
styles,  but  design  more  than  ten  writing  forums  on  the  best  seller  TUESDAYS  WITH  MORRIE  (Mitch  Albom)  and 
related  resources  on  web. 

Topic  discussion  and  practice 

To  encourage  the  student  to  participate  in  the  online  teaching  activity,  we  use  diverse  means  of 
communication  to  promote  the  student’s  initiative.  For  example,  we  assign  students  in  several  teams;  students  in 
each  teach  can  either  exchange  learning  experience  within  the  team,  or  with  those  in  the  other  teams.  This  requires 
the  teacher  to  prepare  some  topics  in  advance  and  make  them  known  on  the  notice  board  a week  or  two  weeks 
ahead.  The  student  can  voice  his  or  her  view  in  English  freely,  or  raise  some  questions  that  he  or  she  is  unable  to 
answer,  which  needs  the  teacher  to  give  instructions  or  make  a summary  in  the  end.  In  addition,  we  regard  the 
student’s  participation  as  important  as  the  writing  quality  as  a part  of  evaluation  on  the  student’s  learning 
proficiency. 

If  the  student  is  required  to  express  his  or  her  view  on  a problem  or  write  a short  article,  the  task  can  be 
accomplished  independently  by  the  student  him-  or  herself,  or  by  the  team  as  a whole.  The  student  is  learning 
autonomously,  while  the  teacher  is  supervising  remotely.  As  writing  is  a matter  of  practice  more  than  just  learning 
some  knowledge,  we  lay  special  stress  on  designing  the  practice  part. 

3)  Literature  support 

To  make  full  use  of  Internet’s  advantage  of  global  information  sharing,  we  super-connect  the  websites  related  to 
writing,  including  information  related  to  the  designed  topics,  online  libraries  and  E-dictionaries  for  the  sake  of 
facilitating  the  learning  process  of  the  student. 

4)  Master  page  of  teaching 

The  online  mast  page  of  teaching  can  not  only  fuse  the  teaching  curriculum,  references  and  various  resources  as  an 
entity,  but  give  each  student  a workable  space  to  introduce  his  or  her  experience  in  writing.  We  advocate  a 
coordinative  and  competitive  online  learning  mode;  the  student  can  create  an  E-periodical  to  publish  their  exercise 
writing  works  and  make  comments  on  the  writing  course  they  are  engaged  in.  The  students  may  be  asked  to  answer 
the  following  questions: 

If  you  were  the  teacher,  how  would  you  design  this  course?  What  suggestions  would  you  make  to  those 
who  are  scheduled  to  join  the  course?  Is  there  any  part  that  you  expected  to  have  in  the  course  but  actually  is 
missing?  Will  you  recommend  this  course  to  your  friends?  Why?  Answers  to  these  questions  would  help  the  teacher 
improve  design  of  the  course [2]. 

Teaching  strategy  design  of  the  online  English  writing  course 

Online  English  writing  teaching  depends  on  modern  educational  technology  and  web  technology.  On  the 
other  hand,  by  drawing  on  the  experience  in  conventional  writing  teaching,  it  follows  the  process  from  oral 
description  to  literal  writing,  and  from  imitation  to  creation,  leading  the  student  in  a heuristic  and  step-by-step  way 
to  turn  what  he  or  she  sees  or  hears,  from  the  easy  to  the  difficult,  from  the  simple  to  the  complex,  and  from  the 
superficial  to  the  deep,  into  a series  of  thinking,  and  finally  turn  what  is  in  mind  into  a written  form.  In  this  way, 
problems  encountered  by  the  student  in  English  writing  can  be  solved  more  effectively  in  a non-native-English 
environment. 
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The  form  of  teaching  organization 

Unlike  the  conventional  teaching  form,  the  classroom  in  online  teaching  is  only  seen  in  the  first  class  when 
the  student  is  registered  into  the  course  and  in  the  last  class  when  the  course  ends.  In  the  first  class,  the  teacher 
makes  a general  introduction  of  the  course,  purpose  of  the  course,  teaching  methodology,  criteria  for  evaluation,  and 
the  function  of  the  website  and  how  to  use  it.  In  the  last  class,  the  student  displays  his  or  her  achievements  from  the 
course  while  the  teacher  makes  a comprehensive  evaluation  on  each  student.  Obviously,  online  teaching  is 
conducted  in  teams,  especially  in  the  form  of  individual  on-machine  maneuver.  Online  interaction  in  a sense  hatches 
the  concept  of  individualized  learning.  The  web  setting  also  provides  a new  mode  for  team  learning  by  breaking 
through  the  concept  of  students  being  in  close  proximity  to  each  other  in  the  classroom;  the  student  is  allowed  to 
choose  any  other  student  as  a partner  of  the  team  to  discuss  the  writing  problem  so  that  the  student  can  have  a true 
understanding  about  the  meaning  of  “net  neighbor”.  This  will  instill  an  significant  impact  on  the  student’s  way  of 
thinking [4]. 

Teaching  methodology 

Teaching  in  web  settings  becomes  more  flexible.  The  teacher  is  able  to  display  the  teaching  content  by 
means  of  net  broadcast,  or  provide  the  student  with  a virtual  milieu  for  writing  drill,  or  recommend  cooperative 
learning  between  students  by  giving  instructions  to  individual  students  and  revisions. 

i.  Teacher’s  creation  of  writing  situations  by  means  of  the  web  broadcast  function 

Ample  evidence  has  demonstrated  that  the  extent  of  success  in  teaching  often  depends  on  the  initiative  of 
mental  activity  of  the  learner  in  the  teaching  process.  Interest  is  the  impetus  of  learning  on  the  part  of  the  student. 

We  can  publish  teaching  materials  and  background  information  of  popular  topics  by  means  of  the  web  broadcast 
function  to  tempt  the  student’s  interest  in  writing,  or  create  a situational  atmosphere  related  to  the  teaching  content 
where  the  student  is  able  to  experience  a virtual  but  true  living  space. 

ii.  The  student  makes  use  of  the  virtual  environment  such  as  the  writing  forum  that  web  provides  to  do 
writing  drills 

Through  exchanging  ideas  with  other  students  by  means  of  the  writing  forum,  the  student’ s sense  of 
creation  is  stimulated.  In  such  a virtual  environment,  the  student  is  freed  from  psychic  stress  from  being  questioned 
directly  so  that  his  or  her  thinking  is  activated  to  the  maximum.  He  or  she  is  not  just  receiving  instructions  passively 
but  thinking  actively  and  expressing  ideas  freely. 

iii.  The  teacher  gives  writing  instructions  to  individual  students  through  internal  E-mail. 

We  often  deal  with  “divergence”  passively  in  teaching  a conventional  writing  course.  As  a matter  of  fact, 
divergence  is  an  objective  existence.  The  right  and  active  way  of  solving  this  problem  is  to  go  into  the  internal 
psychic  environment  of  individual  students  and  help  them  develop  the  strong  points  and  avoid  the  weak  points  by 
using  appropriate  curricula  so  as  to  perfect  the  knowledge  structure  of  the  student  and  develop  his  or  her  writing 
ability.  Internal  E-mail  provides  us  with  an  ideal  channel  of  going  into  the  thinking  process  of  the  student  in  writing. 
The  teacher  is  able  to  get  timely  information  about  how  frequently  the  student  is  online,  the  writing  material  and 
score  by  means  of  internal  E-mail. 

iv.  The  student  can  make  use  of  the  writing  databank  by  means  of  web  technology. 

Thinking  training  in  conventional  writing  teaching  mainly  includes  examining  the  topic,  and  selecting  and 
organizing  materials  around  the  topic.  This  often  leads  to  a tendency  that  all  students  are  thinking  in  the  same  way. 
This  is  not  beneficial  to  foster  the  student’s  interest  in  writing  and  develop  new  individualized  styles  of  writing.  Web 
technology  has  changed  the  conventional  one-way  teaching  mode,  whereas  it  provides  the  student  with  a space 
where  he  or  she  can  think  freely  and  autonomously  [5]. 

v.  Comment  and  revision  making  of  typical  writings  by  taking  advantage  of  web  monitoring  and  broadcast 
functions 

Educational  evaluation  is  a feedback  and  correction  system  between  the  teacher  and  the  student.  The 
efficiency  of  writing  revision  as  a means  of  evaluation  should  be  raised  to  such  a level  that  the  student  can  lose  no 
time  to  find  his  or  her  gain  and  loss  in  the  process  of  writing  so  that  his  or  her  ability  of  making  revisions  in  writing 
is  improved.  We  first  deliver  some  typical  sentences  and  paragraphs  to  the  students  by  means  of  the  web  broadcast 
function,  and  then  the  teacher  and  the  student  work  together  to  complete  the  task  of  correction  in  a warm  and 
cheerful  atmosphere,  where  the  student  airs  their  views  freely.  In  the  process  of  correcting  the  writings  of  other 
student,  his  or  her  writing  ability  is  being  raised  imperceptibly.  Urged  by  the  sense  of  performance,  the  student  is 
willing  to  show  off  his  or  her  writing  in  front  of  other  students  and  savor  happiness  of  success;  he  or  she  will  not  feel 
embarrassed  in  public  because  of  his  or  her  illegible  handwriting. 
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Implementation  of  the  online  English  writing  course  in  practice 

We  set  up  the  online  English  writing  course  in  Grade  2001,  2002,  2003  and  2004  doctorial  graduates  in  our 
school,  and  each  course  lasts  12  weeks  in  40  teaching  hours.  Through  the  online  course  and  the  whole  teaching 
process,  the  teacher  savors  a new  teaching  model,  and  the  student  is  offered  an  opportunity  to  develop  their 
capabilities  and  improve  their  writing  skills.  Also,  the  student  learns  good  qualities  from  others  and  strengthens 
mutual  ties  between  them  in  the  interactive  process.  Most  of  them  have  had  more  confidence  in  their  writing  ability. 
Indeed,  the  online  English  writing  course  is  beneficial  to  the  teacher  as  well  as  to  the  student. 

Achievements  in  teaching  practice 

Undoubtedly,  the  most  important  achievement  is  that  all  students  have  made  measurable  progress  in 

writing. 

i.  The  number  of  articles  is  increasing  dramatically  and  the  number  of  words  of  an  article  is  usually 
between  300-400. 

ii.  ii.  The  students  can  write  a variety  of  articles  including  narratives,  expository,  descriptive  and 
argumentative  essays.  Grammatical  errors  are  reduced  greatly,  the  language  becomes  more  vivid, 
and  the  sentence  structures  are  more  complex. 

iii.  iii.  The  teaching  content  is  more  varied,  and  discussion  and  idea  exchange  go  further  into  the 
center  rather  float  on  the  surface. 

Summary  of  web  course  practice 

Online  courses  should  be  fused  into  the  whole  network  of  the  other  existing  courses.  The  purpose,  the 
curricular  content,  teaching  scope  and  depth  should  be  defined  more  clearly  and  scientifically,  which  include: 

i.  Defining  the  teaching  goal  of  every  module  on  the  web  page; 

ii.  Defining  the  form  of  expression  on  the  basis  of  the  course  content; 

iii.  Teacher’s  responsibility  of  marking  the  relationship  between  the  web  address  and  the  course 
content,  reminding  the  student  of  problems  that  should  care  for  in  visiting  the  address  and 
indicating  the  goal  to  be  achieved  lest  the  student  should  wander  aimlessly  on  the  web. 

The  key  to  success  is  sufficient  exchange  of  ideas.  Whether  the  teaching  content  is  appropriate  is  also 
reflected  to  some  extent  by  the  amount  of  teaching  hours.  Our  experience  shows  that  5 hours  a week  is  necessary  for 
each  student  spending  on  the  writing  course,  which  include: 

i.  1 hour  for  learning  about  the  content  of  the  course  and  reading  references; 

ii.  2-3  hours  for  online  writing  drills,  including  some  online  academic  activities 

iv.  It  is  important  for  the  managerial  personnel  to  coordinate,  supervise  and  manage  the  whole  online 
teaching  process.  The  managerial  personnel  should  work  together  with  the  teacher  to  give  the 
student  a clear  explanation  about  the  course,  examine  whether  the  student  has  qualification  of 
selecting  the  course,  and  find  and  solve  problems  that  emerge  in  the  teaching  process. 

v.  Assignment  design  and  correction  are  more  important  for  online  teaching.  Assignment  design 
should  be  based  on  the  teaching  purpose  and  workable  on  the  web. 

vi.  The  style  and  method  of  examination  should  be  adjusted.  The  online  examination  no  longer  uses 
the  conventional  written  form;  rather  it  is  conducted  in  the  form  of  groups:  each  group  presents  the 
topic  discussion  orally  first,  and  then  makes  it  into  a web  page  or  publishes  it  on  the  website  in  the 
form  of  PPT 


Summary 

In  summary,  a sound  design  is  the  key  to  success  of  online  teaching.  Undoubtedly,  use  of  web  technology 
is  a revolution  in  foreign  language  writing  teaching  in  that  it  breaks  through  the  time  and  space  limit  of  the 
conventional  writing  teaching.  The  web  environment  provides  the  student  with  a realm  of  displaying  their 
personality  traits  and  arouses  the  students'  initiative  in  writing.  The  abundant  literature  resources  and  language  tools 
on  Internet  make  writing  a pleasant  and  creative  maneuver.  A survey  shows  that  94%  of  the  graduates  engaged  in 
the  course  agree  that  the  online  English  writing  course  as  a new  teaching  model  is  better  than  the  conventional 
English  writing  course;  and  100%  of  the  students  believe  that  spreading  this  kind  of  online  language  teaching 
courses  has  practical  or  far-reaching  significance.  Like  any  other  teaching  methods,  online  teaching  is  by  no  means 
perfect,  and  there  are  various  problems  that  need  to  be  studied  and  solved.  Nevertheless,  we  believe  that  online 
language  teaching  is  a sector  that  warrants  more  attention  and  efforts  in  future  language  teaching. 
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Abstract 

Now,  in  its  third  phase,  this  project  management  simulation  has  added  a third  traditional  graduate  project 
management  class  to  serve  as  liaison  between  traditional  undergraduate  and  online  graduate  classes  and  serves  as 
senior  management  and  mentors  to  undergraduates.  The  online  class  serves  as  a client  that  generates  RFP's  for 
undergraduates.  The  findings  from  this  simulation  will  be  discussed,  as  well  as  recommendations  for  establishing 
collaboration  between  undergraduate  and  graduate  students  in  a hybrid  environment. 

Introduction 

Western  Illinois  University  offers  B.S.  and  M.S.  degrees  in  Instructional  Technology  and 
Telecommunications  (ITT).  The  need  for  this  class  arose  from  a perceived  separation  that  existed  between  our 
undergraduate  students,  who  were  instructed  in  traditional  or  web-enhanced  classes,  and  our  graduate  students,  who 
completed  most  of  their  courses  online,  many  of  whom  were  at  a distance.  It  was  felt  that  the  usual  community  of 
practice  that  existed  between  undergraduate  and  graduate  students  was  weakened. 

The  undergraduate  and  graduate  programs  in  ITT  both  offer  an  instructional  project  management  course. 
The  undergraduate  version  of  the  course  focuses  more  on  working  as  part  of  an  instructional  design  team.  The 
graduate  course  focuses  more  on  developing  management  skills  for  instructional  designers. 

These  two  courses  seemed  like  an  ideal  vehicle  for  developing  collaboration  between  the  undergraduate 
and  graduate  communities.  In  Spring,  2003,  the  first  case  study  was  conducted  using  one  team  of  graduate  students 
in  the  graduate  class  and  the  entire  undergraduate  project  management  class.  Since  that  time,  the  model  has  been 
expanded  in  three  later  case  studies,  including  one  that  paired  undergraduate  and  graduate  students  taking  an 
introductory  course  in  instructional  technology.  In  Spring  2004,  the  model  was  expanded  to  include  an  entire 
traditional  undergraduate  class  on  instructional  project  management  and  an  online  graduate  class  on  the  same  topic 
(Hemphill  and  Spencer,  2004).  From  the  results  of  this  study,  it  was  felt  that  there  needed  to  be  more  direct  face-to- 
face  (F2F)  interaction  between  undergraduates  and  graduate  students,  so  a third  graduate  section  was  added  that  met 
F2F  in  a traditional  classroom  setting,  but  also  had  online  access  to  the  undergraduates,  themselves,  and  the 
graduates  in  the  online  section. 

It  was  felt  that  such  a model  would  allow  a direct  mentoring  experience  for  F2F  graduate  students,  while 
providing  interaction  in  the  process  for  online  students.  It  also  gave  the  undergraduates  experience  with  dealing  with 
clients  (online  graduate  students)  and  senior  management  (F2F  graduate  students).  The  primary  goal  was  to  develop 
a community  of  practice  that  extended  beyond  physical  boundaries  and  to  explore  a cognitive  apprenticeship  model 
for  both  undergraduate  and  graduate  students. 


Background 

Theoretical  and  research  underpinnings 

Instructional  project  management  is  a complex  discipline  requiring  a broad  range  of  skills.  It  combines 
project  management  skills,  as  defined  by  the  Project  Management  Institute  (1997),  and  the  instructional  design 
process.  Today’s  workplace  is  increasingly  requiring  instructional  project  management  skills  of  instructional 
designers.  Evolving  technology  is  causing  the  elements  requiring  management  to  be  even  more  complex  and 
interactive.  Communication  challenges  become  more  demanding  as  management  occurs  over  distances  rather  than 
at  one  location  (Litchfield  & Keller,  2002).  Effective  communication  is  at  the  heart  of  skillful  instructional  project 
management.  Devising  methods  to  teach  these  important  skills  effectively  can  be  a challenge.  It  is  essential  for 
later  transfer  to  job  performance  that  the  job  context  be  integrated  into  learning  (Clark  & Mayer,  2003).  For  this 
reason,  the  authors  chose  to  provide  their  students  with  the  same  experiences  that  they  would  encounter  in  the 
working  world,  supplemented  with  group  interaction  and  coaching.  This  included  exposure  and  appropriate  practice 
at  the  various  levels  of  interaction  and  role  modeling  involved  in  instructional  project  management. 


234 


The  cognitive  apprenticeship  model  (Collins,  Brown,  & Newman,  1989)  has  been  used  in  online  mentoring 
models  (Dorneich  & Jones,  1997).  It  has  been  found  to  be  an  effective  model  for  preparing  students  for  entering  the 
professional  workplace.  Graduate  students  have  been  used  for  the  mentoring  of  undergraduate  students  (Russell  & 
Daugherty,  2001;  Koals,  2000). 

Nevertheless,  developing  a community  of  practice  can  be  somewhat  difficult  to  develop  and  maintain  in 
online  classes  and  even  traditional  classes  (Dorneich  & Jones,  1997).  Another  challenge  is  trying  to  develop  a 
cognitive  apprenticeship  program  across  graduate  and  undergraduate  students  (Russell  & Daugherty,  2001;  Koals, 
2001).  The  solution  involves  finding  a common  project  or  projects  with  appropriate  levels  of  mastery  for  the 
students  (Clark  & Mayer,  2003).  The  project  roles  must  be  authentic,  yet  attainable  for  students’  level  of  proficiency 
and  understanding  of  the  field  and  real-world  experiences. 

Need 

Online  students  sometimes  feel  disconnected  from  the  learning  community  on  campus.  They  do  connect 
with  each  other  through  discussion  groups  and  collaborative  projects,  but  do  not  always  feel  connected  to,  or  even 
know,  the  undergraduates  in  the  same  program. 

The  graduate  students  need  experience  managing  others  since  they  will  normally  be  entering  the  work  force 
at  management  or  supervisory  levels.  Conversely,  the  undergraduate  students  will  be  entering  more  production  level 
positions.  They  need  to  learn  to  work  with  others  and  understand  how  on-the-job  mentoring  can  be  an  effective  way 
to  learn. 

Today,  more  actual  projects  use  virtual  project  management  approaches  that  utilize  telecommunication 
tools  to  allow  project  team  members  to  work  at  a distance  in  a shared,  virtual  work  space.  Both  undergraduate  and 
graduate  students  need  experience  working  collaboratively  in  such  an  environment. 

The  previous  studies  with  ITT  courses  had  limited  involvement  of  graduate  students  with  the  undergraduate 
students.  It  was  felt  by  having  a F2F  group  of  graduate  mentors,  this  would  serve  to  develop  a better  cognitive 
apprenticeship  model  and  serve  as  a liaison  between  the  undergraduates  and  online  graduate  students. 

Also,  there  was  a need  for  a common  goal  for  all  groups.  This  was  initiated  by  the  RFP  first  developed  by 
the  graduate  online  class  and  then  implemented  by  the  undergraduate  class.  This  allowed  all  members  to  see  the 
development  of  the  project  from  conception  to  completion. 

Method 

Context 

The  study  occurred  in  the  spring  of  2005  at  Western  Illinois  University.  All  three  courses  were  offered 
through  the  Instructional  Technology  and  Telecommunications  Department.  The  undergraduate  course  (ITT  330) 
was  offered  three  days  a week,  as  was  the  F2F  graduate  course  (ITT  565).  A second  section  of  ITT  565  was  offered 
online.  All  three  courses  used  support  through  the  Blackboard  CMS. 

ITT  330:  Instructional  Project  Management  is  a core  course  in  the  undergraduate  program  for  a B.S.  in  the 
Department  of  Instructional  Technology  and  Telecommunications.  The  course  is  usually  taken  during  the  junior  or 
senior  year  at  WIU  and  is  offered  in  the  spring  semester.  Students  coming  into  the  course  are  expected  to  have  taken 
ITT  310:  Principles  of  Instructional  Design  prior  to  taking  this  course.  This  semester,  the  course  consisted  of  twenty 
four  students. 

ITT  330  concentrates  on  techniques  used  by  training  and  education  specialists,  instructional 
designers,  and  managers  to  plan  a budget,  schedule  a project,  and  lead  a project  team  through  to  successful 
completion.  In-class  discussions  and  take-home  reading  assignments  are  integrated  with  hands-on  experiences  that 
lead  to  the  development  and  production  of  instructional  materials  for  education,  business,  and  industry.  The  course 
is  also  designed  to  help  ITT  students  begin  their  individual  portfolios  needed  for  graduation. 

The  goals  of  the  course  center  around  leading  a group  effort  on  all  or  part  of  a project.  Students 
immerse  themselves  in  the  process  of  identifying  client  needs  through  designing  and  developing  appropriate 
solutions  to  fit  those  needs,  all  in  a group  setting.  During  the  semester,  each  individual  has  the  opportunity  to  lead  as 
the  project  manager  on  a part  of  the  project. 

Rather  than  setting  the  class  up  as  a series  of  project  management  activities,  the  class  was 
designed  to  put  students  into  situations  that  simulate  the  real  world  as  closely  as  possible.  Learners  were  exposed  to 
the  basic  concepts  of  project  management  early  in  the  course,  then  developed  their  skill  as  part  of  fulfilling  a role  on 
a “real”  project. 

ITT  565:  Management  of  Instructional  Technology  is  an  elective  course  that  has  been  offered  in  F2F  and 
online  formats.  It  was  first  offered  online  in  the  spring  of  2003.  It  is  requires  a prerequisite  of  ITT  505:  Foundations 
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of  Instructional  Technology.  Emphasis  is  given  to  project  management,  assessment  of  instructional  needs, 
coordination  of  instructional  design,  production  of  instructional  materials  and  supervision  of  instructional  projects. 

In  contrast  to  ITT  330,  the  emphasis  is  more  on  teaching  management  skills.  This  is  facilitated  by  allowing 
every  student  to  manage  an  instructional  project.  Many  of  the  participants  in  ITT  565  have  real-world  working 
experiences  in  schools,  higher  education,  and  corporate  settings.  Discussions,  reaction  papers,  project  management 
simulation  activities,  and  projects  help  move  the  student  from  learning  simple  principles  and  concepts  to  more 
complex  models  of  project  management,  including  more  adaptive  project  management  models  (Wysocki,  2003). 

There  were  eight  students  in  the  traditional  ITT  565  course.  Eleven  students  participated  in  the  online 
version  of  the  graduate  course. 


Subjects 

A team  of  graduate  students  role-played  as  instructional  developers  of  a web-based  project  for  a client,  the  ITT 
department.  The  undergraduates  served  as  subcontractors  to  the  graduate  students. 

A team  of  students  from  the  graduate  class  developed  a request  for  proposal  (RFP)  for  the  development  of  the 
university  departmental  web  site.  Teams  of  undergraduates  responded  to  the  RFP  with  their  own  proposals.  The 
graduate  team  selected  the  best  undergraduate  proposal.  All  of  the  face-to-face  undergraduate  class  worked  on 
completion  of  the  winning  project. 

The  communication  process  used  face-to-face  planning  sessions,  teleconferences,  synchronous  online 
discussions,  and  asynchronous  discussions,  both  within  and  between  the  graduate  and  undergraduate  teams. 
Undergraduate  students  were  able  to  play  various  roles  in  the  project  management  process.  Both  the  undergraduate 
teams  and  the  graduate  team  were  able  to  gain  experience  in  negotiating  with  a client.  The  graduate  students 
developed  communication  skills  for  managing  projects  when  the  various  stakeholders  are  at  a distance  and 
telecommunication  is  essential. 

Course  Design 

Undergraduate  Course 

The  undergraduate  course  was  divided  into  three  main  components.  The  first  was  a four- week  introduction 
to  project  management.  The  second  was  a group  project  that  consisted  of  writing  a response  to  a Request  For 
Proposal  (RFP).  This  took  four  weeks.  The  third  was  a group  project  that  consisted  of  designing  and  developing  a 
project  based  on  an  approved  proposal.  The  remainder  of  the  16  weeks  was  devoted  to  this  project. 

Based  on  feedback  from  previous  instructors  and  students,  one  of  the  biggest  hurdles  in  the  course  has  been 
convincing  students  of  the  need  to  work  in  groups.  Often  they  are  more  comfortable  working  alone  on  projects  and 
have  not  had  much  experience  in  group  work.  In  addition,  many  students  are  concerned  about  having  a non- 
contributing member  in  their  group  affecting  their  own  performance  and  grade. 

The  first  part  of  the  course  was  an  introduction  to  project  management,  using  lectures  and  class  activities. 
This  part  of  the  course  lasted  four  weeks  and  had  two  main  goals.  First,  the  course  provided  an  introduction  to 
general  concepts  of  project  management.  These  concepts  included  defining  a project,  organizing  for  the  long-term, 
and  using  project  management  with  instructional  design.  Second,  students  needed  to  become  accustomed  to  working 
in  groups.  Each  class  period  consisted  of  group  activities.  The  activities  called  for  groups  of  varying  sizes  and 
required  students  to  be  in  groups  with  different  people  each  day.  The  benefit  of  this  approach  was  to  overcome  both 
the  instructor  and  students  challenges  and  concerns  regarding  working  in  groups.  These  activities  provided  low-risk 
opportunities  for  students  to  become  acquainted  with  one  another  and  to  discover  their  own  strengths  and 
weaknesses  regarding  group  work. 

Once  the  class  finished  the  general  introduction  to  project  management,  the  course  changed  focus.  Now  it 
was  time  to  put  students  in  situations  where  they  had  to  apply  the  principles  of  project  management. 

Group  Project  1 was  designed  to  provide  this  opportunity.  Students  took  on  roles  within  a simulated 
organization  that  allowed  them  to  apply  these  principles.  They  worked  with  the  ITT  565:  Management  of 
Instructional  Technology  class.  ITT  330  students  took  on  the  role  of  a design  team  for  a large  organization.  The  ITT 
565  class  served  as  senior  managers  for  the  same  organization. 

For  Group  Project  1,  the  ITT  330  class  was  divided  into  three  groups  of  eight  people  each.  Students  formed 
their  own  groups.  Each  group  was  divided  into  two  sub-groups:  business  management  and  content  management. 
Each  sub-group  selected  their  project  manager.  Groups  were  encouraged  to  create  an  identity  for  themselves.  They 
accomplished  this  by  creating  names  for  their  groups  and  devising  slogans  and  logos  to  represent  themselves.  Group 
names  were  very  positive  and  proactive  sounding:  Superior  Management  Consultants,  Allstars  Web  Development, 
and  Acme  Industries.  Establishing  this  identity  helped  to  promote  teamwork  within  each  group. 

Once  groups  were  established,  they  were  ready  for  their  first  project.  Group  Project  1 began  when  senior 
managers  (ITT  565  students)  presented  a Request  For  Proposal  (RFP)  to  the  groups.  The  RFP  concerned  redesigning 
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and  revising  the  Instructional  Technology  & Telecommunications  web  site.  Each  group  was  to  submit  their 
proposal,  with  the  best  proposal  “winning”  the  contract  to  redesign  the  ITT  web  site. 

Each  group  was  responsible  for  writing  a proposal.  The  two  project  managers  for  each  team  were 
responsible  for  ensuring  their  group  completed  their  requirements  for  the  RFP.  For  each  group,  the  business  sub- 
group researched  and  wrote  the  proposal  components  dealing  with  budgeting,  schedules,  and  responsibilities,  while 
the  content  sub-group  concentrated  on  determining  the  information  that  the  new  web  site  would  contain.  Having  two 
project  managers  for  each  group  allowed  more  people  to  fulfill  the  project  manager  role.  In  addition,  two  senior 
managers  from  ITT  565  were  assigned  to  each  group  to  provide  guidance  and  answer  questions. 

As  each  group  concentrated  on  writing  their  proposals,  they  communicated  regularly  with  the  senior 
managers  on  their  progress.  They  wrote  weekly  status  reports  and  met  regularly  with  respective  senior  managers. 
Group  Project  1 culminated  with  each  group  presenting  their  proposal  to  the  entire  ITT  565  class. 

The  ultimate  outcome  for  Group  Project  1 was  a project  proposal  in  response  to  an  RFP.  Each  of  the  three 
groups  successfully  completed  their  proposals  on  time  and  made  presentations  to  the  senior  management  group.  The 
groups  were  expected  to  present  their  proposals  in  a professional  manner,  including  dressing  appropriately  and 
providing  documentation. 

Setting  up  the  groups  and  the  RFP  as  a real  world  simulation  created  competition  between  the  groups. 
Students  in  the  groups  were  motivated  in  that  their  projects  were  not  just  going  to  be  evaluated  and  graded  by  the 
instructor.  The  groups  were  also  motivated  to  succeed  because  they  were  drafting  a proposal  that  would  be  evaluated 
by  their  “superiors,”  with  the  goal  that  one  of  the  proposals  would  be  selected  for  development.  The  possibility  that 
their  proposal  would  result  in  their  design  actually  being  adopted  by  the  ITT  department  provided  additional 
motivation. 

The  senior  management  team  from  the  ITT  565  project  management  class  was  very  helpful  in  evaluating 
the  proposals  and  providing  feedback  to  the  groups.  One  challenge  that  arose  was  that  the  ITT  330  groups  were  very 
confident  of  the  quality  of  their  proposals.  Although  each  group’s  proposal  was  very  strong,  they  each  also  had 
limitations  and  flaws.  The  graduate  class  did  a good  job  of  identifying  both  the  strengths  and  weaknesses  of  the 
proposals.  However,  the  ITT  330  class  had  a difficult  time  accepting  the  negative  feedback,  even  though  this 
feedback  was  appropriate.  It  was  important  for  the  instructor  to  provide  a real-world  context  for  the  comments  from 
the  graduate  class. 

The  groups  were  evaluated  on  this  project  in  the  following  ways.  First,  their  proposals  were  evaluated  by 
the  instructor  as  well  as  the  senior  management  team  according  to  how  well  they  fulfilled  the  requirements  of  the 
RFP.  This  included  proposing  an  appropriate  solution,  developing  and  following  a timeline,  assigning 
responsibilities  with  a Work  Breakdown  Schedule  (WBS),  and  establishing  a budget.  In  addition,  group  members 
evaluated  each  other  according  to  how  well  they  fulfilled  their  responsibilities  within  the  group. 

The  second  group  project  was  a larger  effort  in  which  the  class  was  given  a large,  multi-faceted  project  to 
design  and  develop  a web  site  for  the  ITT  department.  While  the  first  group  project  was  a relatively  small  effort, 
Group  Project  2 was  designed  to  provide  the  opportunity  to  take  a project  proposal  and  actually  deliver  a finished 
solution. 

Group  Project  2 began  with  a completed  project  proposal  given  by  the  senior  management  to  the  ITT  330 
project  teams.  The  project  consisted  of  designing  and  developing  a new  web  site  for  the  ITT  department.  This  site 
was  to  focus  on  marketing  the  department  to  prospective  students,  including  incoming  freshmen  and  transfer 
students.  The  site  needed  to  showcase  the  technical  abilities  of  the  department  and  so  would  consist  of  various 
multimedia  elements  such  as  video  and  Flash  animations. 

Students  were  placed  into  entirely  different  groups  for  this  project.  This  was  done  for  two  reasons.  First, 
this  project  required  four  different  project  teams  rather  than  three,  one  for  content,  video,  web  development,  and 
Flash.  Second,  the  instructor  felt  that  it  was  important  to  break  up  cliques  and  conflicts  that  formed  in  the  first  group 
project.  Also,  changing  groups  forced  students  out  of  their  comfort  zones  and  more  accurately  reflected  what 
happens  in  the  real  world. 

The  new  groups  were  formed  by  surveying  students  to  see  what  their  skills  were  in  the  various  technical 
areas.  Based  on  their  responses  and  observations  from  Group  Project  1,  the  instructor  placed  students  in  each  of  the 
four  groups.  Once  the  groups  were  identified,  each  group  selected  a project  manager.  The  project  managers  for  each 
group  could  not  be  one  who  served  as  a project  manager  for  Group  Project  1.  In  addition,  it  was  necessary  to 
identify  one  person  to  serve  as  the  project  manager  for  the  entire  project.  This  person  would  coordinate  the  efforts 
amongst  the  groups  and  be  the  main  liaison  between  the  ITT  330  class  and  the  senior  managers  in  ITT  565. 

Each  group  was  required  to  establish  their  own  WBS  and  timeline.  The  group  project  managers  were 
responsible  for  assigning  tasks  within  the  group  and  ensuring  their  groups  met  project  deadlines.  The  class  project 
manager  worked  closely  with  the  group  project  managers  to  make  sure  the  entire  project  was  on  schedule.  Often, 
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one  group’s  deadline  was  dependent  on  another  group.  For  example,  the  web  group  was  able  to  create  a prototype  of 
the  site,  but  had  to  wait  for  the  content  group  in  order  to  create  the  final  pages. 

The  project  had  three  milestones,  each  of  which  had  to  be  presented  to  and  approved  by  the  senior 
management  team.  The  first  was  a WBS  and  accompanying  timeline.  Each  group  provided  this  to  the  class  project 
manager,  who  then  combined  them  into  a master  schedule  and  Gantt  chart.  The  second  was  a working  prototype  and 
scripts.  The  third  was  a final  presentation  with  a completed  finished  web  site.  Each  group  was  able  to  complete  their 
tasks  on  time,  mainly  because  of  the  interdependence  between  the  groups. 

Online  Graduate  Course 

Project  Flow 

The  collaborative  project  was  constructed  in  three  phases:  general  introduction  to  project  management,  the 
initial  group  project  to  produce  an  RFP  and  respond  with  a proposal,  and  the  final  project  to  produce  an  RFP  and 
respond  with  a proposal  and  final  product. 

The  initial  stage  of  the  project  for  all  three  classes  was  to  provide  a basic  grounding  in  the  principles  of 
project  management.  This  was  provided  at  a more  advanced  level  for  the  graduate  565  classes.  Since  not  all  students 
in  the  565  classes  had  the  undergraduate  330  class  (not  all  had  been  ITT  undergraduates),  it  was  necessary  to  review 
basic  concepts  in  the  context  of  the  role  of  managers.  The  goals  for  this  initial  stage  are  presented  in  Figure  1. 

Figure  1 . Goals  of  general  introduction 


The  next  stage  of  the  model  (Figure  2)  was  to  have  the  online  graduate  class  develop  an  RFP  for  a project 
that  was  related  to  the  final  product,  but  broader  in  scope.  The  outcome  for  the  undergraduate  class  was  that  they 
would  respond  to  the  RFP  with  a proposal.  They  would  not  continue  the  project  any  further. 

The  F2F  graduate  class  served  a dual  role.  They  provided  feedback  to  the  online  group  and  selected  the 
final  RFP  that  was  presented  to  the  undergraduate  class. 

They  also  reviewed  the  proposals  and  provided  feedback  to  the  undergraduate  class,  and  also  provided 
feedback  into  the  selection  of  the  winning  proposal  by  the  instructor. 
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Figure  2.  Goals  of  project  1 
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• Write  a project  proposal  in  response  to  an  RFP 

• Develop  a WBS  and  project  timeline 

• Determine  a project  budget 

• Deliver  a presentation  of  the  project  proposal 

565  Online  Goals 

• Write  an  RFP  that  clearly  specifies  project  goal 

• Revise  RFP  based  on  input  for  F2F  565 

565  F2F  Goals 

• Review  and  provide  feedback  to  online  565  on  initial  RFP 

• Guide  and  provide  feedback  to  330  in  responding  to  RFP 
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The  final  stage  was  to  develop  a realistic  RFP,  have  the  330  class  respond  to  edit  and  then  produce  the  final 
product  based  on  the  selected  proposal.  This  second  RFP  was  smaller  in  scope  and  involved  a redesign  of  the  ITT 
undergraduate  website.  The  RFP  was  developed  by  three  different  groups  in  the  565  online  class.  The  best  RFP  was 
selected  and  presented  to  the  330  class  by  the  565  F2F  class.  The  330  class  was  then  divided  into  three  groups  and 
responded  to  the  RFP  with  proposals.  The  best  proposal  was  selected  with  input  from  the  F2F  565  class.  The  330 
class  then  worked  on  the  final  product,  based  on  the  selected  proposal.  The  F2F  565  served  as  advisory  senior 
management  throughout  the  development  of  the  project.  Figure  3 shows  the  respective  goals  of  the  final  stage, 
Project  Two. 
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Figure  3.  Goals  of  project  2 
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330  Goals 

• Develop  a WBS  and  project  timeline 

• Develop  specific  project  deliverables  according  to  proposal  specifications 

• Meet  project  deadlines 

• Cooperate  within  and  between  project  teams 

• Respond  appropriately  to  client  needs  and  senior  management  feedback 

565  Online  Goals 

• Develop  an  RFP  that  was  realistic  for  330  capacity  and  timeline 

• Revise  RFP  from  feedback  from  565  F2F 

• Review  progress  on  project  by  330 

565  F2F  Goals 

• Review  initial  RFP  and  provide  feedback  to  online  565 

• Present  final  RFP  to  330 

• Select  final  proposal  by  330 


Results 

Undergraduate  course 

Evaluations  for  Group  Project  2 were  different  from  Group  Project  1.  In  this  case,  the  instructor  evaluated 
individuals  according  to  how  well  they  accomplished  tasks  given  to  them  on  the  WBS.  In  addition  to  instructor 
evaluation,  heavy  emphasis  was  placed  on  peer  evaluations.  Peer  evaluations  were  done  in  three  steps.  First,  each 
group  member  evaluated  the  other  members  of  their  group  and  their  group  project  manager.  Second,  each  group 
project  manager  evaluated  their  group  members,  the  other  project  managers,  and  the  class  project  manager.  Third, 
the  class  project  manager  evaluated  the  group  project  managers. 

Group  Project  2 did  a good  job  of  demonstrating  the  complexities  of  how  large  projects  with  multiple 
deliverables  are  completed.  It  showed  how  important  it  is  to  be  able  to  work  together  in  a group  setting. 
Communication  skills  and  conflict  management  were  essential  to  the  success  of  the  project.  Each  group  had 
concentrated  on  delivering  their  projects  within  the  deadlines  they  had  established.  Unfortunately,  the  quality  of  the 
final  project  was  not  at  a level  that  would  allow  the  site  to  be  adopted  by  the  department.  Part  of  this  was  due  to  the 
time  limits  of  the  class  itself.  Given  additional  time,  the  senior  managers  would  be  able  to  provide  appropriate 
feedback  so  that  the  groups  would  be  able  to  revise  their  products  as  needed. 
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Online  graduate  course 

Eleven  (11)  students  enrolled  in  ITT  565  online.  Ten  (10)  of  these  students  responded  to  the  pre  and  post 
survey  given  in  the  course.  The  pre-survey  was  issued  during  the  first  week  of  course,  and  the  post-survey  in  the  last 
week  of  the  course. 

The  collaboration  effort  appeared  to  improve  overall  satisfaction  of  the  course,  as  well  as  enjoyment  in 
working  in  groups,  and  changed  their  belief  in  a collaboration  environment.  In  the  pre-survey,  90%  agreed  to  the 
following  statement:  “Collaboration  is  conducive  to  learning”  - 100%  agreed  in  the  post  survey.  In  the  pre-survey, 
80%  agreed  to  the  statement:  “I  enjoy  working  with  others  on  a project”,  90%  agreed  afterward.  Ninety  percent 
(90%)  agreed  in  the  post  survey,  compared  to  80%  in  the  pre  survey  that  being  a project  team  member  was  helpful 
to  their  learning  experience.  100%  of  students  in  the  post  survey  stated  that  being  familiar  with  project  management 
in  the  ITT  program  at  WIU,  and  the  IT  field  was  important.  This  is  compared  to  90%  pre-survey. 

Some  changes  that  were  made  this  semester  from  the  past  format,  was  that  instead  of  the  client  being  a 
professor,  the  client  was  another  class.  They  exchanged  RFPs,  while  allowing  a practice  RFP  first.  This  gave  each 
group  a chance  to  respond  to  each  of  the  areas:  web,  print  and  multimedia.  The  graduate  students  acted  as  guides  to 
the  undergraduate  students  instead  of  being  clients  that  made  demands  of  them.  This  helped  with  more 
communication  between  the  two  groups,  and  less  bad  feelings  that  the  undergrads  were  being  controlled.  It 
promoted  more  of  a common  goal  for  all  three  groups.  The  online  group  had  people  in  their  group  that  knew  people 
in  the  face-to-face  course,  which  also  gave  them  physical  “representatives”  for  the  undergraduate  course.  Previously 
the  graduates  and  undergraduates  only  interacted  in  person  on  a very  limited  basis.  Graduate  students  also  came  to 
the  undergraduate  course  regularly  so  they  could  meet  about  the  RFPs  and  the  project. 

Each  group  (graduate  and  undergraduate)  completed  a project  together,  and  one  on  their  own.  This  gave 
them  almost  complete  freedom  on  one  project  in  which  they  could  choose  to  do  individually  or  in  groups,  and  some 
sense  of  control  with  the  second  project.  Undergraduate  students  were  not  “forced”  to  take  the  advice  of  the  graduate 
students,  which  provided  less  resentment. 

Deadlines  were  also  expanded  to  allow  more  interaction  between  all  groups.  Due  dates  were  delayed  until  a 
few  weeks  into  the  semester,  instead  of  the  first  week  or  two.  This  allowed  more  conversation  and  interaction 
between  all  the  groups. 

All  11  students  responded  to  the  final  evaluation.  Most  students  enjoyed  working  with  undergraduate 
students,  and  listed  it  as  the  favorite  part  of  the  course.  ITT  565  online  students  offered  the  following  suggestions  for 
improvement:  better  communication  between  instructors  of  ITT  330  and  ITT  565,  and  one  posting  place  for 
activities.  Each  group  provided  a supportive  role  to  the  other,  which  contributed  to  a productive  learning 
environment  and  everyone  feeling  a part  of  something  bigger. 

From  the  instructors’  perspective,  this  was  a valid  learning  experience  for  the  students,  and  also  the  faculty. 
It  was  a learning  experience  to  see  the  two  graduate  courses  and  the  undergraduate  course  work  together,  however, 
working  together  as  faculty  members  for  each  course  was  also  a learning  experience  from  the  teaching  perspective. 

Face-to-face  graduate  course 

The  F2F  graduate  course  was  very  involved  throughout  the  final  process  of  development.  Graduate  students 
were  assigned  to  work  directly  with  the  330  development  teams.  The  entire  F2F  class  was  present  for  the  final 
presentation  of  the  final  product.  One  member  of  the  online  class  was  also  able  to  be  present  for  this  final  session. 

The  F2F  565  class  completed  a final  report  on  their  experience  in  the  class,  on  their  own  development 
projects,  and  on  the  collaboration  project  with  the  other  two  classes.  The  comments  about  the  collaboration  were 
generally  very  positive. 

The  students  liked  the  experience  of  being  senior  managers.  They  felt  they  gained  experience  on  what  to 
expect  from  a design  team,  how  to  lead  people  through  a project  life  cycle,  how  to  critique  design  team  projects,  and 
generally  how  to  be  a good  manager. 

They  also  felt  it  was  a good  experience  for  the  undergraduates  and  provided  those  students  with  a project 
that  was  both  fun  and  authentic.  The  F2F  565  enjoyed  the  meeting  and  presentations  with  the  330  students.  They 
learned  the  need  to  be  specific  in  terms  of  proposals,  or  the  final  product  might  not  meet  client  expectations.  Their 
communications  face-to-face  and  online  with  330  and  online  with  the  online  565  class  helped  to  improve  their 
communications  skills.  This  was  especially  important  for  international  students  for  whom  English  was  not  their 
native  language. 

Many  of  the  F2F  565  students  appreciated  the  ability  to  follow  up  the  project  status  online  and  on  their  on 
time.  A couple  of  students  commented  that  they  would  have  liked  to  have  had  more  online  discussions,  but  enjoyed 
the  F2F  interaction. 
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All  F2F  graduate  students  saw  the  experience  as  a very  positive  experience  and  a high  point  of  the  class. 

For  those  that  had  330  as  undergraduates,  they  did  not  see  565  as  a repeat  of  the  undergraduate  class,  but  as  a natural 
continuation  of  their  professional  development.  They  felt  they  were  able  to  grow  in  the  role  of  project  manager, 
rather  than  just  as  an  instructional  design  team  member. 

Discussion 

This  was  the  most  successful  implementation  of  this  collaboration  model  so  far.  The  experience  was  rich 
and  rewarding  for  the  undergraduate  and  online  graduate  students.  While  the  online  graduate  students  did  learn  from 
the  experience,  work  needs  to  done  on  improving  communications  with  the  undergraduate  project  team  members. 
The  online  collaborative  discussion  board  can  be  improved  to  make  communications  clearer. 

The  next  implementation  of  this  model  will  focus  on  maintaining  many  of  the  same  strategies  that  were 
used  here.  Reporting  between  the  online  graduate  and  undergraduate  classes  will  be  enhanced  to  improve 
communications  and  allow  the  online  students  to  feel  more  involved  and  more  a part  of  the  learning  community. 

There  are  some  suggestions  for  similar  collaborative  projects  can  be  drawn  from  this  study.  First,  the  nature 
of  the  project  must  be  authentic,  not  made-up  work.  Second,  roles  must  be  clearly  defined  roles  for  each  group. 
Third,  communication  must  be  between  the  groups,  both  online  and  face-to-face.  Finally,  having  a traditional 
graduate  class  that  can  serve  as  a liaison  between  the  undergraduate  and  online  graduate  students  can  be  very 
powerful.  Hybrid  versions  of  the  graduate  classes  could  also  help  serve  in  this  role  as  long  as  there  is  direct  face-to- 
face  interaction  at  various  times  in  the  process. 
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Abstract 

This  paper  analyzes  the  function  and  the  responsibilities  of  online  instructors,  also  named  “facilitators” . 
The  job  title  of  online  instructor  can  be  viewed  as  a new  leadership  in  a Web-based  or  an  online  environment. 

Online  instructors  not  only  deliver  knowledge,  provide  information  and  technical  supports  to  learners,  but  also  try 
to  meet  learners  varied  needs.  In  the  beginning,  the  paper  examines  learners  ’ different  needs  during  online  learning, 
especially  the  socialization  needs,  which  are  recognized  as  the  most  influential  factor  to  the  success  of  online 
learning.  The  paper  then  discusses  the  Meacher  Theory,  proposed  by  Hughes  ( 2002 ),  to  explain  how  to  apply  this 
leadership  for  supporting  learners'  socialization  needs.  Finally,  the  paper  concluded  that  it  is  necessary  to  generate 
a new  set  of  skills  requirements  for  online  instructors,  for  the  current  E- society.  A later  case  study  will  be 
implemented  to  further  describe  the  expectations  for  the  online  instructor  job. 


Introduction 

The  role  of  the  online  instructor  is  important  in  different  group  sizes  of  collaborative  learning  activities 
(Dillenbourg,  1999).  Online  instructors  are  usually  named  “facilitators”  (Dillenbourg,  1999;  Handy,  2000;  Kulp, 
2000;  Salmon,  2003).  They  are  not  only  responsible  for  facilitating  knowledge  in  an  online  setting,  but  also  taking 
care  of  learners’  socialization  needs  (Bonk,  Kirkley,  Hara  & Dennen,  2001;  Salmon,  2003).  According  to  Bonk, 
Kirkley,  Hara  and  Dennen  (2001),  online  instructors  play  their  roles  in  four  action  areas:  pedagogy,  social 
interaction,  management  and  technology.  Additionally,  Salmon  (2003)  presented  a five- stage  model  of  teaching  and 
learning  online,  which  highlights  the  learners’  socialization  needs  in  the  online  setting.  Hughes  (2002)  developed  the 
Meacher  Theory  to  analyze  the  leadership  roles  of  the  online  instructor.  This  paper  uses  the  Meacher  Theory  to 
propose  a new  leadership  skill  set,  including  its  function  and  its  responsibility  for  learner  socialization  needs  in  a 
Web-based  or  online  learning  environment. 


Learners’  Socialization  Needs 

The  literature  is  filled  with  examples  of  new  online  socialization  needs.  In  a Web-based  or  an  online 
environment,  which  are  often  learner-centered  (Kirkley,  Savery  & Grabner-Hagen,  1998;  Dillenbourg,  1999;  Kodali, 
1998),  because  learners  have  different  online  learning  difficulties,  they  have  different  learning  needs,  such  as  needs 
to  discuss  course-related  content  with  instructors  and  with  other  learners  (Kirkley,  Savery  & Grabner-Hagen,  1998). 
Learners  also  need  to  be  active  participants  in  an  open-ended  activity  (Kulp,  2000).  Additionally,  they  need  to  obtain 
group-  and  technology  related  skills  (Handy,  2000). 

Further,  Salmon’s  (2003)  five-stage  model  of  teaching  and  learning  online  discussed  learners’  different 
online  learning  needs  in  different  stages.  Following  are  the  examples  of  learners’  needs  within  in  the  five  stages: 

(1)  Access  and  Motivation  Stage — learners  need  information  and  technical  support  to  access  to  the  Internet 
and  strong  encouragement  to  become  actively  involved.  It  is  the  online  instructors  who  must  provide  some 
individual  technical  support  and  model  the  value  of  online  learning  (Salmon,  2003).  This  is  consistent  with  Kodali’ s 
view  of  online  instructors  as  the  learners’  “facilitator,  collaborator,  guide,  and  motivator”  (Kodali,  1998,  p.iv) . 

(2)  Online  Socialization  Stage — learners  need  some  opportunities  to  build  their  online  identities  and  need 
to  find  other  learners  to  interact  with.  They  need  some  incentives  to  change  their  learning  relationships  and  become 
concerned  about  the  cultivation  of  trust  and  mutual  respect  in  a new  learning  culture  which  reflects  diverse  opinions 
(Salmon,  2003).  To  facilitate  the  development  of  the  learning  culture,  Bonk,  Kirkley,  Hara  and  Dennen  (2001) 
suggested  that  online  instructors  should  provide  many  social  activities,  such  as  providing  greeting  statements  at  the 
beginning  of  the  front  page,  sharing  personal  digital  pictures  and  profiles,  and  encouraging  learners  to  give 
comments  on  weekly  discussion  to  each  other. 

(3)  Information  Exchange  Stage — learners  need  to  maintain  their  learning  interests  and  improve  their 
learning  skills.  A learner  needs  to  be  an  independent  learner  confidently  involving  in  interaction  with  others,  and 
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able  to  manage  possible  information  overload  and  time  constraints  (Salmon,  2003).  Examples  of  information 
exchange  support  are  stated  by  Bonk,  Kirkley,  Hara  and  Dennen,  (2001)  who  suggested  that  online  instructors  not 
only  play  pedagogical,  social  and  technological  roles,  but  also  have  managerial  (or  administrative)  roles  to  provide 
instant  feedbacks,  to  monitor  learners’  learning  progress,  and  to  ensure  that  learners  are  constructing  their 
knowledge  through  online  discussion  boards  or  E-mail. 

(4)  Knowledge  Construction  Stage — learners  need  to  have  lots  of  opportunities  to  appreciate  other 
learners’  perspectives,  to  create  their  own  meanings  about  some  ideas,  to  solve  real-situated  problems,  to  collect 
everyone’s  contributions,  and  to  build  knowledge  in  groups  (Salmon,  2003).  This  is  an  ongoing  activity  that  was 
initiated  in  stage  1 and  stage  2.  Althauser  and  Matuga  (1998)  stated  that  scaffolding  instruction  is  needed  to  help 
learners  effectively  absorb  learning  materials  and  find  a way  to  apply  them  in  daily  life. 

(5)  Development  Stage — learners  need  support  to  develop  their  meta-cognitive  skills  in  order  to  evaluate 
their  learning  processes.  Because  in  the  online  environment  they  are  more  responsible  for  their  learning,  they  must 
be  encouraged  to  reflect  upon  the  different  interactive  activities  and  the  social  network  in  the  online  learning 
environment  (Salmon,  2003).  Dillengourg’s  (1999)  stated  that  collaborative  learning  happens  in  a social 
psychological  process,  which  contains  cognitive  mechanisms. 

Based  on  Salmon’s  five  stages,  we  can  view  the  first  stage  as  a fundamental  socialization  stage.  Within  the 
preparation  stage,  Handy  (2000)  implied  that  preparing  a comfortable  computer  supported  collaborative  learning 
environment  through  three  stages — Think  it  through,  Getting  out  of  the  way,  and  Not  really  necessary — requires 
the  deliberate  design  of  routines  that  support  collaboration.  On  the  other  hand,  the  second  stage  can  be  viewed  as  the 
critical  socialization  stage.  Kodali  (1998)  suggested  that  promoting  interaction,  giving  feedbacks,  facilitating 
discussion,  creating  learner-centered  activities,  ensuring  flexibility  and  communicating  with  learners  are  online 
instructors’  teaching  strategies  for  fulfilling  learners’  socialization  needs  (Kodali,  1998).  Other  stages  are  like 
expansive  explanations  of  learners’  socialization  needs,  such  as  (a) synchronous  information  exchange  tools, 
individual  or  group  knowledge  construction  opportunities,  and  meta-cognition  development  activities. 

To  sum  up,  we  can  clearly  point  out  that  the  most  important  learners’  need  is  socialization.  Bonk,  Kirkley, 
Hara  and  Dennen  (2001)  stated  that  socialization  is  vital  to  learners  in  an  online  learning  environment.  Other 
researchers  have  begun  to  study  whether  different  group  sizes,  grading  criteria  or  interactive  activities  have  different 
influences  on  learners’  socialization.  For  example,  Althauser  and  Matuga  (1998)  regarded  that  out-of-class  small- 
group  electronic  conferencing  (which  is  synchronous  communication)  can  create  a better  learning  culture  and  help 
learners  truly  share  knowledge  together.  Alternatively,  Kirkley,  Savery,  and  Grabner-Hagen  (1998)  found  that  E- 
mail  (which  is  asynchronous  communication)  can  increase  discussion  duration  between  learners.  What  is  the  nature 
of  learning?  Specifically,  what  is  the  nature  of  online  learning?  We  may  directly  answer  this  question  with 
“socialization”.  This  means  that  no  matter  what  kinds  of  learning,  such  as  case-based  or  problem-based  learning, 
learners’  socialization  needs  are  the  main  tasks  that  online  instructors  have  to  take  care  of  throughout  the  whole 
process  of  online  learning. 


Leadership  Supports  Socialization  Needs 

How  do  online  instructors  make  use  of  their  teaching  strategies  or  leadership  to  support  the  learners’ 
socialization  needs?  What  kinds  of  teaching  strategies  or  leadership  do  the  online  instructors  have?  Strang  in  1958 
had  stated  that  group  leadership  has  great  impact  on  academic  performance  and  the  effectiveness  of  group 
participation  (Flannery,  1994).  Kodali’ s (1998)  research  found  that  online  instructors  use  some  instructional 
components  of  Reigeluth  and  Merrill’s  General  Model  of  Instructional  Design,  specifically  the  ‘methods’ 
component  with  it  organization,  delivery  and  management  variables  and  the  leadership  that  is  required  to  execute 
them.  Additionally,  Hughes  (2002)  conducted  an  action  research  study  for  examining  the  effectiveness  of  the 
Meacher  Theory  of  Leadership  for  helping  herself  be  an  appropriate  instructional  leader  in  both  traditional  face-to- 
face  and  online  classrooms.  We  may  find  that  a new  set  of  skill  description  for  online  instructor  is  necessary  in  the 
current  E-society  because  this  job  is  mainly  about  supporting  online  learning,  especially  learners’  socialization 
needs.  Hughes  applied  the  Meacher  Theory  of  Leadership  (Figure  1)  as  an  example  framework  of  strategies  to 
discuss  how  online  instructors  can  use  leadership  to  support  learners’  socialization  needs  in  an  online  learning 
environment. 

“Meacher”  means  to  combine  the  concepts  of  mothering  and  teaching  perspectives;  so  this  theory  implies 
female  nurturing  and  caring  functions  (Hughes,  2002).  In  her  study,  Hughes  found  herself  often  applying  feminist, 
moral,  and  democratic  leaderships  in  online  classes.  She  also  applied  other  kinds  of  leadership  theory,  such  as 
transformational,  transactional,  and  servant  leaderships.  Concerning  feminist  leadership  (or  women  leadership), 
Regan  and  Brooks  in  1995  regarded  that  some  female  online  instructors  might  be  able  to  easily  build  trust  between 
learners,  and  even  efficiently  solve  problems  together  by  using  their  nurturing  skills  (Story,  1997).  This  efficiency  is 
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a basis  for  transactional  leadership,  which  relates  to  online  instructors’  managerial  or  organizational  skills 
independently  or  in  teams.  The  feminist  leadership  is  also  related  to  online  instructor’s  moral  leadership,  which 
might  influence  learners’  values  and  beliefs;  in  addition,  the  application  of  democratic  leadership  establishes  basic 
rules  or  policies  for  learners  interacting  with  each  other  (Hughes,  2002).  Burns  in  1978  stated  that  transformational 
leadership  is  also  related  to  the  moral  leadership,  since  it  can  increase  online  instructors’  motivation  and  enhance 
their  relations  with  learners  during  transforming  processes,  such  as  changing  learners’  sense  of  personal 
responsibilities  and  encouraging  learners  to  contribute  their  efforts  more  during  discussion  (Hughes,  2002).  In 
Figure  1,  we  can  find  that  she  already  combined  the  transformational  and  transactional  leaderships,  because  she 
regarded  that  these  two  processes  happen  at  the  same  time.  Servant  leadership,  which  means  being  in  charge  to 
make  conscious  choice  by  serving  people,  is  another  one  that  Hughes  applied,  recognizing  her  role  made  her 
responsible  for  learners’  interactions  in  the  online  learning  environment. 

There  is  no  dominant  leadership,  so  online  instructors  should  apply  various  leaderships  based  on  different 
situations  (Hughes,  2002).  These  six  leadership  theories  can  be  viewed  as  teaching  strategies,  helping  online 
instructors  complete  the  fundamental  and  critical  social  preparation  and  maintenance  tasks.  For  example,  thinking 
about  what  moral  leadership  means  can  assist  the  instructor  to  identify  and  fulfill  the  learner’s  social  needs.  Through 
identifying  moral  topics  such  as  the  need  to  be  honest  and  reliable,  the  instructor  is  guided  to  include  reflective 
prompts  on  these  topics.  This  paper  suggests  that  applying  these  six  theories  can  result  in  building  an  appropriate 
social  culture  for  the  online  learning  environment. 

Conclusions  and  Further  Research 

Facing  different  learner  socialization  needs  in  a Web-based  or  online  learning  environment,  the  Meacher 
Theory  of  Leadership  can  be  one  set  of  skills  requirements  for  online  instructors.  This  paper  has  pointed  out  the 
importance  of  building  a complete  skill  set,  which  include  social,  psychology  and  other  possible  dimensions.  This 
skill  set  can  help  online  instructors  examine  themselves,  for  example,  in  which  part  of  online  teaching  skills  have  to 
be  concerned  or  need  to  be  trained.  In  the  later  case  study,  the  researchers  will  continue  review  literature  and  studies 
for  creating  the  skill  sets  by  describing  their  functions  and  responsibilities.  The  researchers  will  then  conduct  a 
survey  to  investigate  which  online  instructional  skills  that  K-12  instructors  have.  Those  K-12  instructors  will  be  the 
research  participants,  who  must  have  online  teaching  experiences.  After  the  survey  data  has  been  collected,  the 
researcher  will  report  what  kinds  of  skills  are  currently  obtained  by  what  percentage  of  online  instructors.  The 
participants’  feeling  and  expectations  of  being  online  instructors  can  also  express  on  the  survey  and  through  the 
follow-up  interviews.  In  the  end  of  the  research  report,  the  researchers  will  also  describe  other  expectations  for  the 
online  instructor  job.  In  conclusion,  it  is  necessary  to  create  the  skill  sets  for  the  online  instructors  for  the  need  of 
online  socialization  in  the  current  E-society. 
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Introduction 

The  Appalachian  College  Association  is  an  organization  of  35  private  liberal  arts  colleges  in  the  central 
Appalachian  Mountains,  which  includes  more  than  3,250  faculty  and  serves  over  39,000  students  in  the  states  of 
Kentucky,  North  Carolina,  Tennessee,  West  Virginia,  and  Virginia.  The  Association  provides  expanded  and 
enhanced  learning  opportunities  to  faculty  and  students  for  the  purpose  of  serving  the  educational  and  economic 
well-being  of  the  people  and  communities  of  central  Appalachia.  Through  the  generous  support  of  the  Andrew  W. 
Mellon  Foundation,  ACA  has  encouraged  collaboration  among  faculty  on  the  various  ACA  campuses  for  several 
years.  The  project  described  in  this  paper  represents  a continuation  of  the  AC  A/Mellon  Foundation  support. 

Lee  University  is  a private,  comprehensive  university  located  in  Cleveland,  Tennessee,  in  the  foothills  of 
the  Appalachian  Mountains.  Over  the  past  eighteen  years,  Lee  has  become  the  second  largest  private  institution  in 
Tennessee  and  the  largest  in  the  Appalachian  College  Association.  Lee’s  enrollment  has  more  than  tripled  from 
1,214  students  in  1986  to  3,930  students  in  fall  2005.  Two  full-time  faculty  and  6 part-time  faculty  serve  the  CIS 
area. 

Lincoln  Memorial  University  (LMU)  is  a private,  liberal  arts  university  located  55  miles  north  of  Knoxville, 
Tennessee,  near  the  border  with  Kentucky  and  Virginia.  LMU  offers  more  than  30  academic  majors,  3 master’s 
degree  programs,  and  the  Educational  Specialist  degree.  Fall  2005  enrollment  is  2,802.  One  full-time  and  three 
part-time  faculty  serve  the  CIS  area. 

Milligan  College  is  a private,  liberal  arts  college  in  northeast  Tennessee  offering  more  than  25  academic 
majors  and  3 master's  degree  programs.  Milligan  has  an  enrollment  of  964  students.  Two  full-time  faculty  serve  the 
CIS  area. 


Why  Collaborate? 

The  three  private  institutions  described  above,  separated  from  each  other  by  a distance  of  approximately 
150  miles,  needed  to  offer  elective  courses  in  Computer  Information  Systems  (CIS).  All  three  institutions  faced  the 
same  problems:  offering  students  a variety  of  courses,  especially  current  topics  in  the  CIS  field,  with  a limited 
number  of  full-time  CIS  faculty.  Among  the  three  schools  there  are  only  five  full-time  faculty  positions  offering  99 
courses  per  year.  This  means  that  there  is  a significant  shortfall  in  full-time  instructor  availability.  These 
institutions  rely  on  nine  part-time  faculty  to  help  satisfy  the  demand.  This  means  that  each  of  the  institutions  has  a 
very  difficult  time  trying  to  offer  their  students  current  and  relevant  courses  in  the  field.  Traditionally,  smaller 
institutions  such  as  these  have  had  difficulty  attracting  and  retaining  terminally  degreed  faculty  because  their  salaries 
do  not  match  those  offered  by  the  business  world  or  a larger  university.  Each  institution  has  difficulty  obtaining 
sufficient  enrollment  for  special  topics  and  elective  courses. 

Collaboration  allowed  these  faculty  members  to  combine  their  talents  and  expertise  in  the  subject  matter  in 
teaching  the  course.  A sense  of  collegiality  was  felt  and  professional  development  was  enhanced  as  the  faculty 
worked  together  to  develop  and  administer  the  course.  Collaboration  also  brought  to  students  on  each  local  campus  a 
diversity  of  culture  and  experience  and  helped  build  a more  robust  enrollment  for  the  course. 

Two  of  the  participants  in  this  collaboration  had  previous  experience  with  ACA- sponsored  course 
collaboration.  During  2002  and  2003,  Lee  University,  LMU,  and  the  University  of  Charleston  offered  online  two 
courses  in  Computer  Operating  Systems  for  students  on  all  three  campuses.  About  30  students  participated  in  each 
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of  the  courses,  which  included  both  online  instruction  and  operating  system  lab  exercises  on  each  of  the  three 
campuses. 


How  We  Collaborated 

The  first  step  in  the  collaborative  effort  was  to  secure  support  from  the  academic  administration  at  each  of 
the  institutions.  Meetings  were  held  with  the  Vice  President  for  Academic  Affairs  on  each  campus.  Issues 
discussed  included  ownership  of  the  course,  financial  responsibility  for  tuition,  course  workload  assignment,  course 
enrollment  levels,  course  hosting  responsibilities  and  technical  support  for  the  course  management  system. 
Administrative  support  for  collaborating  on  this  course  was  granted  by  all  three  institutions  at  the  Academic  Vice 
President  level.  It  was  agreed  that  each  institution  would  enroll  its  own  students,  collect  its  own  tuition,  and  record 
its  own  students’  grades. 

The  three  institutions  settled  on  a special  topics  course,  a course  in  Information  Security,  because  this  was  a 
very  relevant  topic  at  the  time.  The  three  instructors  met  several  times  at  a central  location  or  via  telephone 
conference  calls  to  discuss  the  development  of  the  course.  After  reviewing  several  texts,  the  faculty  chose  an 
appropriate  text  for  the  course.  During  these  meetings,  the  duties  and  responsibilities  of  teaching  and  administering 
the  course  were  divided  among  three  instructors.  Angel  was  chosen  as  the  course  management  system  due,  in  part, 
to  the  fact  that  Lee  University  had  just  adopted  it  and  had  the  server  space  and  technical  support  for  the  system. 
Training  on  the  course  management  system  was  conducted  to  equip  the  faculty  with  the  skills  necessary  to  manage 
the  course. 


Course  Development 

The  faculty  agreed  on  a framework  for  the  course  management  system  layout  so  that  each  chapter  had  a 
consistent  look  and  feel  for  the  students.  The  textbook  had  12  chapters  and  each  instructor  was  assigned  to  'lead' 
every  third  chapter,  making  only  four  chapters  for  which  any  one  instructor  was  responsible.  Their  responsibilities 
included  answering  student  questions,  grading  the  chapter  assignments  dropped  in  the  digital  drop  box,  and  creating 
appropriate  test  questions  for  the  chapters.  One  of  the  instructors  still  had  full-time  responsibilities  as  Chief 
Information  Officer  on  his  campus;  the  other  two  instructors  carried  full-time  teaching  loads  in  addition  to  this 
course. 

The  faculty  developed  a common  calendar  for  the  course  by  collapsing  the  academic  calendars  at  the  three 
colleges  into  one.  Students  were  expected  to  work  through  each  individual  college  break  by  adhering  to  the 
combined  calendar  for  the  course.  The  faculty  worked  together  to  develop  the  syllabus,  define  the  student 
requirements,  set  up  assignments  and  determine  the  exam  style.  It  was  determined  that  exams  would  be  proctored, 
with  each  faculty  member  meeting  his/her  own  students  at  their  home  institution  to  complete  the  three  online  exams. 
Each  faculty  member  was  responsible  for  creating  one  of  the  three  exams  with  input  from  the  other  faculty  members 
about  appropriate  questions  to  include  on  the  exam. 
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Enrollment  in  the  Course 

Twenty-two  students  enrolled  in  the  course.  As  the  chart  below  shows,  17  percent  of  the  students  were 
female  and  83  percent  were  male. 


Grade  Classification 


As  the  chart  above  shows,  the  course  enrollment  consisted  of  55  percent  seniors,  39  percent  juniors  and  6 
percent  sophomores.  There  were  no  freshmen  in  enrolled  in  the  course. 

Delivering  the  Course 

Each  campus  held  an  organizational  meeting  during  the  first  week  of  the  semester  to  introduce  the  course 
and  to  make  sure  the  22  students  enrolled  in  the  course  could  access  the  course  management  package.  The 
instructors  explained  all  policies  and  procedures  of  the  course  at  this  time.  Each  chapter  in  the  course  contained 
lectures,  two  online  discussion  questions  (students  had  to  respond  to  one  question  per  chapter),  and  one  written  case 
assignment  per  chapter  which  the  students  completed  and  dropped  in  a digital  drop  box. 
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Gender  Diversity  in  Online  Course 


A major  research  paper  was  also  a part  of  the  class.  To  encourage  the  students  to  think  about  this  paper 
before  the  last  minute,  we  required  them  to  summarize  fifteen  journal  articles  (two  per  week)  to  be  submitted  to  the 
digital  drop  box.  These  summaries  were  only  one-half  page  long  with  the  citation.  Done  correctly,  the  student 
developed  the  paper  over  the  course  of  the  semester  from  the  journal  summaries.  The  instructor  at  the  student’s 
home  institution  graded  both  the  journal  articles  and  the  research  paper.  One  school  met  as  a group  to  hear  student 
presentations  on  their  research  papers. 

There  was  an  online  discussion  forum  set  before  each  of  the  three  examinations  to  allow  students  to  ask 
questions  about  the  chapters  on  which  they  would  be  tested.  All  three  instructors  were  present  in  each  of  the  online 
forums,  which  were  open  to  the  students  for  a period  of  one  hour.  This  hour  was  announced  well  ahead  of  time  and 
if  a student  could  not  be  present,  they  were  encouraged  to  submit  any  questions  ahead  of  time  and  return  to  the 
online  discussion  forum  at  their  convenience  to  review  the  instructor  responses. 

Each  institution  met  with  their  students  during  one  or  two  available  meeting  times  during  the  exam  week  to 
administer  and  proctor  the  exam.  The  instructors  secured  each  exam  with  a password  and  the  students  were  only 
given  the  password  when  they  were  in  the  formal  examination  setting. 

The  course  management  system  provided  the  instructors  and  students  with  a variety  of  tools,  such  as 
internal  communication,  password  security  for  the  course,  personal  assignment  calendars,  drop  boxes  for  assignment 
submission,  online  discussion  forums,  online  testing,  and  student  gradebook  access.  The  course  management 
system  approach  to  distance  learning  provided  the  instructors  and  the  students  with  a convenient  approach  to 
presenting  material,  monitoring  student  progress,  and  providing  immediate  feedback.  Case  study  analysis  and 
online  discussion  forums  were  employed  to  provide  critical  thinking  and  problem  solving  experiences.  The  online 
discussion  forums  and  e-mail  were  used  to  develop  relationships  and  community  between  the  students  and  the 
instructors  as  a means  to  reduce  the  feeling  of  geographical  isolation.  Giving  each  student  the  opportunity  to  lead  an 
online  discussion  session  provided  for  the  development  of  leadership  skills.  A calendar  system  within  the  course 
management  system  provided  the  students  the  opportunity  to  exercise  time  management  skills.  At  the  end  of  the 
course,  students  evaluated  both  the  content  and  the  delivery  method  used  in  the  course. 

Issues  and  Problems 

As  with  any  course,  the  execution  did  not  go  without  a hitch.  Since  the  course  was  listed  on  each  college’s 
schedule  of  classes  as  an  online  course  or  a TBA  course,  there  was  no  uniform  meeting  time  scheduled  to  administer 
the  exams.  Therefore,  multiple  sessions  had  to  be  set  to  proctor  examinations.  A couple  of  students  had  Internet 
access  problems  and  personal  problems,  which  prevented  them  from  completing  assignments  according  to  the 
course  schedule.  Some  students  procrastinated  on  getting  assignments  completed  in  a timely  manner  and  several 
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students  did  not  demonstrate  a high  level  of  critical  thinking  in  completing  the  assignments.  When  the  students 
submitted  exams,  the  spam  filter  blocked  some  of  the  submitted  test  results  and  these  were  retrieved  manually. 

Both  instructors  and  students  encountered  problems  with  the  course  management  package.  Student  lack  of 
understanding  of  the  digital  drop  box  resulted  in  confusion  about  where  to  place  assignments  and  resulted  in  several 
assignments  being  placed  in  incorrect  folders.  In  order  for  graded  assignments  from  the  digital  drop  box  to  be  posted 
automatically  to  the  gradebook,  a grade  could  only  be  assigned  to  the  first  item  a student  dropped  in  the  appropriate 
drop  box.  Occasionally  a student  would  resubmit  an  assignment  for  one  reason  or  another.  They  may  have  made 
editing  changes  or  dropped  in  the  wrong  assignment  originally.  For  a grade  to  show  in  the  gradebook,  instructors 
had  to  place  the  grade  on  the  first  assignment  the  student  dropped  in  the  digital  drop  box  whether  it  was  the  correct 
assignment  or  not. 

The  instructors’  lack  of  understanding  of  the  course  management  system  gradebook  resulted  in  slow 
posting  of  student  grades  and  calculation  of  mid-term  averages.  Students  had  to  submit  assignments  more  than  once 
to  get  them  in  the  correct  drop  box  so  that  the  gradebook  could  calculate  the  grades  correctly.  Students  lacked  an 
understanding  of  discussion  forum  threads,  which  resulted  in  poor  discussions  and  inconsistent  threads. 

Evaluating  the  Course  - The  Student  Perspective 

Students  had  an  opportunity  to  complete  an  online  evaluation  of  the  course  at  the  end  of  the  semester.  As 
an  incentive  to  complete  the  evaluation,  instructors  offered  to  add  points  to  the  final  course  grade  for  those  who 
completed  the  survey.  The  form  used,  which  was  created  by  Virginia  Tech  for  evaluating  on-line  instruction,  was 
used  by  permission  for  this  course.  Eighteen  of  22  students  completed  the  evaluation. 

Six  students  reported  taking  this  course  because  it  was  required  in  their  major,  six  because  it  was  a free 
elective  in  their  major,  five  because  it  was  a general  elective  and  one  because  it  was  a required  course  outside  their 
major. 

Students  evaluated  teacher  performance  by  rating  questions  according  to  the  following  scale:  1)  poor,  2) 
fair,  3)  good,  and  4)  excellent.  The  questions  and  their  average  ranking  by  the  students  on  the  4-point  scale  are  as 
follows: 


Question  Average  score 

1.  Apparent  knowledge  of  the  subject  matter  3.56 

2.  Success  in  communicating  or  explaining  subject  matter  3.22 

3.  Degree  to  which  subject  matter  was  made  stimulating  or  relevant  2.89 

4.  Fairness  in  assigning  grades  3.33 

5.  Concern  and  respect  for  individuals  as  students  3.67 

6.  Administration  of  class  and  organization  of  materials  3.11 

7.  Overall  ratings  of  these  instructors  3.17 

8.  Encouragement  and  management  of  class  interaction  3.22 

9.  Responsiveness  to  inquiries  outside  of  class  3.56 


One  student  commented  he  felt  the  course  was  not  well  organized,  but  blamed  the  course  management 
package  rather  than  the  instructors.  Another  commented  that  the  course  was  extremely  well  organized.  Fifty-six 
percent  of  the  students  who  completed  the  survey  stated  that  the  course  procedures  were  clearly  posted  and  that 
necessary  information  was  received  on  time  from  the  instructors.  Fifty  percent  reported  that  clear  instructions  were 
given  for  using  all  materials.  Several  students  stated  that  they  missed  the  interaction  with  faculty  and  other  students 
that  come  from  a traditional  course  setting.  Some  students  commented  that  they  missed  the  lectures  the  professor 
would  normally  have  given.  Although  discussion  forums  were  opened  before  each  test  with  all  three  instructors 
participating,  the  students  stated  that  they  missed  face-to-face  reviews  for  tests  and  did  not  see  the  discussion  forums 
as  an  acceptable  substitute. 

Students  were  asked  to  rate  questions  related  to  online  instruction  by  indicating  whether  they  1)  strongly 
disagree;  2)  disagree;  3)  agree;  or  4)  strongly  agree  with  the  following  statements. 


Question  Average  score 

I would  recommend  this  online  course  to  others  2.72 

I would  recommend  these  instructors  to  others  taking  an  online  course  3.17 

I would  recommend  that  others  take  a totally  online  course  2.35 
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Comments  on  the  course  evaluation  showed  that  some  students  misinterpreted  the  nature  of  the  course 
thinking  it  dealt  with  security  application  rather  than  security  theory.  Many  students  were  ill  prepared  for  the  rigors 
of  online  learning  and  the  amount  of  responsibility  it  puts  on  the  learner.  The  online  nature  of  the  course  required 
much  more  reading  than  most  students  were  used  to  doing.  Students  also  made  suggestions  for  additional  materials 
that  could  enhance  the  course  such  as  required  completion  of  end  of  chapter  review  questions.  Students  seemed  to 
feel  that  unless  certain  end  of  chapter  exercises  were  specifically  assigned,  they  were  not  at  liberty  to  complete 
them.  Student  evaluations  indicated  online  communication  was  useful  in  the  course,  but  they  needed  more 
orientation  to  the  course  and  to  the  technology  used  in  the  course. 

Evaluating  the  Course  - The  Faculty  Perspective 

The  instructors  met  to  evaluate  the  course  after  it  was  completed.  Several  items  were  identified  as  ways  to 
improve  the  next  course  offering.  Faculty  felt  they  should  have  started  developing  and  preparing  the  course  earlier 
in  the  previous  semester.  Perhaps  more  practice  with  the  course  management  system  would  have  alleviated  some  of 
the  problems  with  both  the  digital  drop  box  and  the  gradebook  portion  of  the  package.  Faculty  felt  that  students 
might  have  performed  better  in  the  threaded  discussions,  in  using  the  digital  drop  box  and  in  meeting  deadlines  if 
they  had  had  more  instruction  about  expectations  and  proper  usage  of  the  course  management  system  at  the 
beginning  of  the  course. 

Video  conferencing  might  have  added  to  the  overall  course  experience  if  a uniform  meeting  time  had  been 
set  for  the  course  in  advance.  The  course  could  have  been  enhanced  by  holding  chat  sessions  to  discuss  chapter 
content  and  other  relevant  issues. 

Because  of  inadequate  numbers  of  full-time  faculty  among  the  three  institutions,  this  collaborative  effort 
served  as  a support  system  for  faculty  with  an  otherwise  heavy  teaching  load.  The  collaboration  also  allowed  the 
faculty  from  the  three  institutions  to  draw  on  the  diverse  interests  and  skills  of  each  other. 

This  course  filled  a need  for  students  to  have  access  to  new  and  developing  topics  in  their  field.  It  also 
provided  them  the  flexibility  to  complete  the  course  around  their  work  and  class  schedules.  This  approach  allowed 
students  from  all  three  institutions  to  participate  in  a class,  rich  in  diverse  viewpoints  and  experiences.  Although 
66%  of  the  students  enrolled  in  the  course  were  white,  non-Hispanic,  the  collaborative  effort  brought  more  ethnic 
diversity  to  the  class  than  teaching  the  course  individually  at  any  one  of  the  institutions. 

Ethnic  Division  in  Online  Course 


Overall,  the  faculty  involved  felt  it  was  a good  experience  both  professionally  and  academically  and  would 
agree  to  repeat  the  course,  or  teach  a similar  course,  with  some  modification.  Other  institutions,  both  small  and 
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large,  could  adopt  similar  strategies  to  solve  problems  of  curricula  with  low  local  student  demand,  no  local  faculty 
expertise,  overextended  faculty,  or  the  need  for  more  diversity  on  their  campuses. 
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Abstract 

This  paper  outlines  the  radical  restructuring  of  a Computer  Science  course  on  Software  Testing.  Authors 
describe  the  course's  evolution  from  a predominantly  lecture-based  format  to  one  based  on  projects  and  in-class 
activities.  This  was  accomplished  by  adding  videotaped  lectures  students  watch  outside  of  class.  All  course 
materials  are  available  under  a Creative  Commons  license  to  academics,  commercial  trainers,  and  self-studiers 
around  the  world. 

Software  testing  is  a relatively  narrow  subfield  of  software  engineering,  but  many  of  the  challenges  we’ve 
faced  in  teaching  it  are  commonplace  in  science,  technology,  engineering,  and  mathematics  (STEM)  education. 
We’ve  developed  an  instructional  approach  that  is  working  well  in  our  academic  setting  and  that  we  believe  can 
extend  to  other  STEM  courses  at  university  and  to  commercial  training. 

Still  a work  in  progress,  the  current  version  of  the  course  differs  from  the  traditional  lecture  course  in  several  ways: 

• We  moved  lectures  out  of  the  classroom.  We  tape  them  in  advance  and  post  them  on  a website.  Students 
watch  lectures  before  coming  to  class.  Frequent  low- stakes  quizzes  motivate  students  to  keep  pace  and 
watch  the  videos. 

• Classroom  contact  hours  are  for  coached  activities,  discussions,  and  student  presentations. 

• Students  apply  the  lessons  of  the  course  to  a well-known  program,  typically  an  open-source  program  in 
development.  We  use  the  same  program  throughout  the  term:  students  join  its  development  team  and  apply 
the  skills  and  knowledge  they  gain  in  the  course  in  ways  that  help  the  project.  The  artifacts  they  create  are 
often  useful  later,  in  job  interviews.  This  offers  many  of  the  same  benefits  of  service  learning,  but  in  a way 
that  is  logistically  simpler  and  less  demanding  for  the  instructor  than  some  service  learning  projects. 

• Students  are  incentivized,  but  not  required,  to  work  in  groups  on  all  types  of  homework.  Students  write 
examinations  on  their  own,  but  most  students  prepare  for  exams  in  teams. 

• We  draw  exam  questions  from  a published  pool,  handed  out  as  a study  guide.  Students  prepare  for  exams 
by  writing  answers  to  the  study  guide  questions.  Because  students  have  had  plenty  of  open-book 
preparation  time  and  opportunity  for  peer-review,  we  can  require  more  carefully  considered,  better  written 
answers. 

• The  course  materials  are  available  to  the  public  (at  http://www.testingeducation.org/BBST)  under  a 
Creative  Commons  license.  Other  university  and  commercial  instructors  use  them  in  their  courses, 
providing  feedback  and  additional  content. 

This  report  describes  the  evolution  of  this  course,  lessons  learned,  and  progress  to  date. 

The  Commercial  Course 

The  course  started  as  in-house  training  based  on  Testing  Computer  Software  (Kaner,  1988),  which  evolved 
into  the  best  selling  book  in  the  field  (Kaner,  Falk,  & Nguyen,  1999).  Kaner  and  Nguyen  developed  a public  version, 
initially  offering  it  to  working  software  testers  and  career-changers  through  the  American  Society  for  Quality  and 
University  of  California  (Berkeley)  Extension  in  1993/1994.  That  course  was  reviewed,  rewritten,  and  extended  by 
several  colleagues,  some  of  them  co-authors  (Kaner,  Bach,  & Pettichord,  2001;  Kaner,  Falk,  & Nguyen,  1993).  The 
course  matured  into  a commercially  successful  practitioner- training  course.  By  2000,  Kaner  had  taught  it  over  100 
times,  many  of  those  at  leading  software  companies.  Nguyen,  his  staff,  and  other  licensees,  taught  it  many  times 
more. 

Commercial  client  expectations  of  courses  like  these  are  limited.  Clients,  fellow  trainers,  and  consulting 
companies  who  sell  training  services  have  repeatedly  advised  us  that  clients  will  be  satisfied  if  each  staff  member  in 
training  gains  one  idea  s/he  considers  useful  from  a three-day  course,  and  very  satisfied  with  one  useful  idea  per  day. 

Clients  expected  lecture- style  instruction  that  covered  a wide  range  of  ideas  and  techniques  at  a crisp  pace. 
Despite  the  popularity  of  this  style  of  course,  full-day  lectures  are  not  well-suited  to  helping  students  develop  new 
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skills  or  develop  a deep  enough  understanding  of  the  material  to  assess  what  testing  technique  is  most  likely  to  be 
effective  under  which  circumstances  (Bligh,  2000).  The  style  and  the  time  compression  flood  the  students  with 
information  and  allow  little  time  for  individual  practice  or  reflection.  Kaner  introduced  some  activities  into  his 
courses — brief,  simple  activities  were  well  received,  but  there  was  often  client  resistance  to  tasks  that  were  more 
challenging  or  clusters  of  tasks  designed  to  approach  a technique  from  several  angles  to  better  support  transfer  from 
classroom  examples  to  workplace  application.  These  activities  had  to  be  done  in  class — students  often  have  to  catch 
up  on  their  other  work  at  night  and  don’t  have  time  to  do  homework  in  short  full-day  courses.  The  more  time  on  in- 
class  activities,  the  fewer  topics  can  be  covered. 

The  Academic  Course 

In  1999,  Kaner  accepted  a professorship  at  Florida  Institute  of  Technology,  with  the  goal  of  deepening 
software  testing  education  beyond  what  can  be  achieved  with  practitioner-level  training.  As  part  of  this  work,  he 
founded  Florida  Tech’s  Center  for  Software  Testing  Education  & Research  (http://www.testingeducation.org).  One 
of  the  Center’s  projects  has  been  transformation  of  the  introductory  software  testing  course  from  commercial  to 
academic.  From  there,  we  are  developing  materials  useful  for  other  university  instructors,  commercial  instructors, 
and  self-studiers.  We  (Kaner  and  Fiedler)  began  collaborating  on  this  project  in  2001. 

University  courses  differ  from  commercial  courses  in  key  ways: 

• The  course  lasts  over  a longer  period  of  time,  introducing  material  at  a much  slower  pace  (a  few  hours  per 
week),  with  plenty  of  time  for  practice  and  reflection  outside  the  classroom. 

• Students  expect  to  do  homework. 

• Students  expect  the  instructor  to  assess  their  work  and  provide  detailed  feedback  on  their  performance. 

• Frequent  assessments  give  the  instructor  objective  feedback  on  the  quality  and  effectiveness  of  instruction 
as  well  as  insight  into  the  knowledge  of  the  students. 

• Students  generally  welcome  coached  practice  in  class  and  appreciate  it  when  the  instructor  takes  extra  time 
to  make  sure  they  are  mastering  key  concepts  in  the  course.  They  are  more  interested  in  the  congruence 
between  the  level  of  support  provided  in  class  and  the  level  of  emphasis  on  the  exam  than  on  the  length  of 
the  checklist  of  topics  covered. 

Kaner  typically  teaches  the  introductory  testing  course  at  Florida  Tech  twice  per  year.  The  opportunity  to 
assess  student  and  instructor  performance  enabled  an  assessment-driven  evolution  of  the  course  from  2000  to  2005. 
Our  key  questions  were: 

• What  is  the  broad  scope  or  subject  matter  of  the  course? 

• What  do  we  want  to  achieve  in  the  course?  (What  are  our  learning  objectives?) 

• How  will  we  assess  the  students’  knowledge  and  progress? 

• What  learning  experiences  will  better  prepare  students  for  these  assessments  and  for  the  workforce? 
Instructional  theory  books  often  present  these  as  sequential  questions.  In  practice,  we  evolved  the  course  on 

all  three  dimensions  in  parallel. 

Subject  Matter 

To  put  the  instructional  approach  in  context,  we  start  with  a few  details  about  the  craft  of  software  testing 
and  the  challenges  doing  and  teaching  it. 

We  see  software  testing  as  a process  of  technical  investigation  of  the  product  under  test  conducted  to 
provide  stakeholders  with  quality-related  information. 

The  classic  definition  of  software  testing  is  “the  process  of  executing  a program  with  the  intent  of  finding 
errors”  (Bumstein,  2003;  IEEE,  1983;  Kaner,  Falk,  & Nguyen,  1999;  Myers,  1979,  p.  5).  Another  second  classic 
definition  is  evaluation  of  the  product  (IEEE,  1990).  A third,  helping  the  project  manager  decide  when  to  put  a 
product  into  production  or  release  it  to  the  public  (Marick,  1997).  There  are  several  others  (Kaner,  Bach,  & 
Pettichord,  2001).  The  common  thread  among  the  definitions  is  that  software  testers  gather  factual  information  about 
the  product’s  quality  on  behalf  of  their  (in-house  or  external)  clients. 

Testing  is  often  focused  on  the  external  behavior  of  the  program,  without  reference  to  the  underlying  code. 
This  is  called  black  box  testing.  In  contrast,  white  box  or  glass  box  tests  are  designed  on  the  basis  of  the 
implementation  details  of  the  code.  One  class  of  glass  box  testing  is  programmer  testing,  testing  done  by  the 
programmer  of  her  or  his  own  code. 

In  deciding  the  curriculum  for  our  recently- accredited  Bachelor’s  degree  in  Software  Engineering,  Florida 
Tech  decided  to  require  students  to  complete  two  testing  courses,  the  first  focused  on  black  box  testing,  the  second 
applying  the  first  course’s  lessons  in  test  design  and  critical  thinking  to  programmer  testing. 
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The  course  described  in  this  report  is  the  black  box  testing  introduction. 

Black  box  testing  involves  a cognitively  complex  collection  of  tasks.  Here  are  a few  of  the  challenges: 

• For  any  nontrivial  program,  the  population  of  possible  tests  is  vast.  A common  example  of  this  presents  a 
simple,  well-structured  program  that  runs  about  20  lines  but  requires  100  trillion  tests.  At  five  minutes  per 
test,  running  all  of  these  tests  would  take  a billion  years  (Myers,  1979).  Along  with  running  tests,  testers 
must  interpret  the  results,  troubleshoot  and  report  failures,  document  their  work,  provide  assessments  and 
other  reports,  and  so  on.  These  are  all  worthy  tasks  but  they  are  unachievable  in  the  time  available. 
Therefore,  testers  must  prioritize  ruthlessly,  choosing  which  worthwhile  tasks  to  skip  or  drastically  subset. 

• Another  problem  that  is  surprisingly  rarely  discussed  in  schools  and  process  improvement  meetings  is  that 
it  is  very  often  very  challenging  to  decide  whether  the  program  actually  passed  or  failed  a test.  The  task  is 
simplified  when  there  is  a complete,  authoritative  specification  but  these  are  rare  in  commercial 
development.  Additionally,  every  test  is  complex.  Consider  this  example.  Suppose  you  test  a program  that 
adds  two  numbers,  and  write  an  automated  test  that  feeds  the  program  under  test  the  numbers  and  checks 
the  sum.  Feed  the  program  2 and  3,  the  result  is  5.  This  looks  like  a passed  test,  but  suppose  that  it  took  the 
program  10  hours  to  compute  the  sum — that  would  be  unacceptable,  but  the  automated  test  tool  might  well 
pass  the  test  (correct  sum)  without  noticing  the  excessive  time.  Every  test  has  many  possible  results  (not 
just  the  sum  and  the  time).  It  can  change  memory,  change  the  state  of  connected  devices,  send  messages, 
and  so  on.  A leading  office  automation  product  manufacturer  showed  Kaner  the  diagnostics  in  its 
firmware.  The  company’s  staff  run  diagnostics  after  every  test  when  doing  some  types  of  automated 
testing.  The  challenge  is  that  there  are  1100  different  diagnostics.  It  is  impractical  to  check  them  all  after 
any  test:  it  takes  too  much  time  and  running  the  diagnostics  changes  the  state  of  the  device  under  test.  Thus 
any  test  can  appear  to  pass  by  some  measure  even  if  it  actually  fails.  We’ve  also  seen  examples  in  which 
the  program  appears  to  fail  a test,  but  is  actually  behaving  correctly.  Assessing  the  results  of  a test — 
especially  deciding  what  to  assess — requires  judgment. 

• There  is  also  broad  disagreement  about  how  to  measure  and  report  testing  progress  (Kaner,  2001 ; Kaner  & 
Bond,  2004). 

Testers  must  choose  and  apply  methods  appropriate  to  their  goals  and  feasible  within  their  project's 
constraints.  The  vast  population  of  possible  tests  takes  the  cost  of  complete  testing  beyond  even  the  broadest 
imaginable  constraints  and  therefore  a key  aspect  of  testing  work  is  prioritization  among  worthy  tasks  that  cannot  all 
be  done.  Another  key  testing  task  is  communicating  investigative  results,  such  as  bug  reports,  project  status  reports, 
and  product  assessments.  These  reports  can  be  controversial,  may  be  taken  as  personal  criticism,  and  especially  for 
bug  reports,  must  be  so  clearly  written  that  someone  else  can  do  detailed  technical  work  on  the  basis  of  them.  In  the 
course  of  their  work,  testers  create  products  of  their  own,  such  as  suites  of  automated  tests  (significant  bodies  of 
code  in  their  own  right)  and  test  documentation  (which  may  run  thousands  of  pages).  Testers  must  be  able  to 
determine  their  own  and  their  stakeholders’  requirements  for  such  work  products. 

Learning  Objectives 

We  were  guided  in  developing  learning  objectives  by  Angelo  & Cross’  (1993)  Teaching  Goals  Inventory. 
The  course  changed  format  several  times,  but  the  objectives  have  been  fairly  stable. 

• Students  should  learn  a variety  of  testing  techniques  and  a variety  of  dimensions  along  which  they  differ — 
and  to  understand  the  variation  well  enough  to  decide  which  technique  is  more  promising  for  a given 
situation,  why,  and  how  to  apply  it. 

• Students  should  apply  (and  further  develop)  communication  skills,  for  example  in  appraising  and  writing 
effective  bug  reports,  reporting  status,  and  analyzing  specifications. 

• Students  should  gain  practical  experience  and  create  artifacts  that  can  help  them  demonstrate  competence 
in  employment  interviews. 

• Students  should  work  in  teams,  developing  and  presenting  material  to  each  other  and  cross-checking  each 
others'  work.  Collaborative  work  is  common  in  industry  (Bransford,  Brown,  & Cocking,  2000)  and 
particularly  common  for  industrial  testers,  whose  primary  work  products  are  documents  that  influence 
others  to  act  (fix  bugs,  change  schedules,  change  designs,  etc.)  and  who  often  review  each  other's  work  for 
impact,  clarity,  and  appropriateness  of  emotional  tone. 

• Students  should  apply  strategic  thinking,  including  prioritizing  tasks,  designing  tests  and  reports  for 
specific  audiences,  and  assessing  requirements  for  complex  testing  tasks. 
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Presentation  of  the  Content 

The  primary  strength  of  the  commercial  course  was  a polished  set  of  lectures  with  many  real-life  anecdotes. 
According  to  Bligh's  (2000)  summary  of  the  literature  on  lecture  effectiveness,  lectures  can  be  as  effective  as  other 
instructional  techniques  for  transmitting  basic  information  about  a topic,  but  they  are  less  effective  than  some  other 
methods  for  teaching  behavioral  skills,  promoting  higher-level  thinking,  or  changing  attitudes  or  values.  In  terms  of 
Anderson  et  al’s  (2001)  learning  objective  taxonomy,  lectures  would  be  most  appropriate  for  conveying  factual  and 
conceptual  knowledge  at  the  remembering  and  understanding  levels.  Our  students  need  to  learn  the  material  at  these 
levels,  but  as  part  of  the  process  of  learning  how  to  analyze  situations  and  problems,  apply  techniques,  and  evaluate 
their  own  work  and  the  work  of  their  peers. 

Carefully  crafted  lectures  offer  several  other  benefits.  Experienced,  charismatic  lecturers  can  share  examples  to  help 
students  learn  complex  concepts,  tasks,  or  cultural  norms  (Ford,  2002;  Hamer,  1999;  Kaufman  & Bristol,  2001). 

Lecturers  can  convey  the  lecturer's  enthusiasm  for  the  subject,  which  improves  student  satisfaction  with  the 
course  (Williams  & Ware,  1977).  They  can  also  organize  and  integrate  a complex  set  of  material.  As  Forsyth  (2003, 
p.  50)  eloquently  puts  it,  "Lectures  provide  the  scholar  with  the  means  of  not  just  disseminating  information  but  also 
transforming  that  information  into  a coherent,  memorable  package.  Scholars,  when  they  reveal  their  unique 
interpretation  of  questions  they  have  spent  years  researching  and  contemplating,  are  an  unmatched  source  of 
information  and  interpretation." 

Rather  than  offering  live  lectures,  we  tape  and  publish  them  on  the  course  website.  This  in  itself  is  hardly 
new.  It  is  common  to  offer  stored  lectures  as  part  of  a distance  learning  program  (Rossman,  1999).  Web-based 
lecture  segments  are  being  used  to  supplement  Computer  Science  courses  (Fintan,  2000).  Studio-taped,  rehearsed 
lectures  with  synchronously  presented  slides  (as  we  do)  have  been  done  and  described  elsewhere  (e.g.,  Dannenberg). 

Students  commonly  report  that  they  prefer  live  lectures  (Firstman,  1983;  Maki  & Maki,  2002).  However,  it 
appears  that,  on  average,  students  learn  as  well  from  video  as  from  live  lecture  (Bligh,  2000;  Saba,  2003).  (We  say 
on  average  because  there  appear  to  be  individual  differences  associated  with  learning  style  or  skill  and  there  are  also 
major  effects  of  online  interaction  on  distance  learning  not  relevant  to  the  live  course  model  we're  working  with.) 

Faculty  often  comment  that  live  lectures  allow  them  to  interact  with  students  in  ways  that  taped  lectures  do 
not.  We  feel  the  same  way,  but  (a)  in  practice,  relatively  few  of  our  students  make  comments  or  ask  questions  in 
class  and  (b)  we  are  sometimes  surprised  by  our  assessment  results  of  what  students  failed  to  learn  from  some  of  our 
lectures.  It  is  true  that  live  lectures  allow  more  interaction  than  taped  lectures  viewable  on  the  web,  but  the  non- 
lecture activities  we  can  develop  in  our  classrooms  provide  far  more  time  and  opportunity  for  interaction  than  live 
lectures. 

• Live  lectures  also  present  some  disadvantages  that  can  be  addressed  in  stored  lectures  (lectures  that  can  be 
accessed  by  students  at  any  time): 

• Short-term  memory  capacity  varies  among  learners,  especially  as  they  age.  Live  lectures  can't  be  rewound: 
what  was  said  is  in  the  past;  if  it  is  not  remembered,  it  is  lost  (Langer,  2002).  In  contrast,  students  watching 
a recorded  video  can — and  do  (He,  Gupta,  White,  & Grudin,  1998) — jump  backwards  in  the  video  to 
review  what  was  said. 

• Students  whose  first  language  is  not  English  often  have  difficulty  keeping  up  with  live  lectures  and  can  also 
benefit  from  the  ability  to  replay  material. 

• Live  lectures  can  become  fragmented  as  students  ask  questions  (and  are  responded  to).  Not  all  student 
questions  are  informative  for  everyone,  and  not  all  questions  come  at  the  most  opportune  time.  In  the  stored 
lectures  we  currently  create,  there  are  no  students  on  tape  and  thus  no  interruptions. 

• Live  lecturers  sometimes  drift  or  ramble.  Stories  are  not  always  directly  relevant  and  some  take  so  long  that 
students  lose  the  message.  Video  lectures  can  be  partially  scripted  (with  a teleprompter),  with  departures 
from  the  script  to  add  spontaneous  remarks.  An  example  that  takes  too  long  can  be  cut  during  post- 
production, or  shortened  in  a retelling  / retaping  of  the  example. 

• Students  can  replay  a stored  lecture  while  studying  for  an  exam  or  working  on  an  assignment,  reviewing 
the  material  in  the  context  of  a specific  problem  or  task  they  are  working  with. 

In  summary,  there  are  advantages  and  disadvantages  to  taped  versus  live  lectures.  At  this  time,  we  are  not 
convinced  that  one  is  better  than  the  other.  However,  by  moving  lectures  out  of  class,  we  create  a semester  of  space 
for  non-lecture  interaction  with  our  students.  By  making  the  lectures  part  of  the  homework,  we  effectively  double 
the  teaching  time  available  to  us  in  the  course. 

We  supplement  the  lectures  with  worked  examples  on  the  website  and  required  readings.  The  examples 
typically  show  detailed  application  of  a test  technique  to  a published  software  product.  Worked  examples  can  be 
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powerful  teaching  tools  (Clark  & Mayer,  2003)  especially  when  they  are  motivated  by  real-life  situations.  They  are 
fundamental  for  some  learning  styles  (Felder  & Brent,  1996).  Exemplars  play  an  important  role  in  the  development 
and  recollection  of  simple  and  complex  concepts  (Brooks,  1978;  Medin  & Schaffer,  1978;  Smith,  2005).  The  lasting 
popularity  of  problem  books,  such  as  the  Schaums  Outline  series,  attests  to  the  value  of  example-driven  learning,  at 
least  for  some  learners.  At  the  moment,  our  examples  are  pure,  non-interactive  text.  We  intend  to  change  these  to 
narrated  videos  that  show  the  test  and  the  failure  in  a (taped)  real-time  demonstration. 

The  required  readings  include  the  course  texts  (currently  Kaner,  Bach,  & Pettichord,  2001;  Whittaker, 

2002)  and  several  published  articles.  We  link  to  the  articles  on  our  website  when  possible,  but  distribute  others  at  a 
Florida  Tech  specific,  password-protected  site  that  restricts  distribution  of  the  copyrighted  articles  to  Florida  Tech 
students  registered  for  the  class. 

Assessment  and  Learning  Activities 

Anderson  et  al.  (2001)  published  an  extensive  revision  and  update  of  Bloom's  (1956)  taxonomy.  Their 
taxonomy  is  two-dimensional,  with  knowledge  and  cognitive  processing  dimensions. 

• On  the  Knowledge  Dimension,  the  levels  are  Factual  Knowledge  (such  as  the  definition  of  a software 
testing  technique),  Conceptual  Knowledge  (such  as  the  theoretical  model  that  predicts  that  a given  test 
technique  is  useful  for  finding  certain  kinds  of  bugs),  Procedural  Knowledge  (how  to  apply  the  technique), 
and  Metacognitive  Knowledge  (example:  the  tester  decides  to  study  new  techniques  on  realizing  that  the 
ones  s/he  currently  knows  don't  apply  well  to  the  current  situation.) 

• On  the  Cognitive  Process  dimension,  the  levels  are  Remembering  (such  as  remembering  the  name  of  a 
software  test  technique  that  is  described  to  you),  Understanding  (such  as  being  able  to  describe  a technique 
and  compare  it  with  another  one),  Applying  (actually  doing  the  technique),  Analyzing  (from  a description  of 
a case  in  which  a test  technique  was  used  to  find  a bug,  being  able  to  strip  away  the  irrelevant  facts  and 
describe  what  technique  was  used  and  how),  Evaluating  (such  as  determining  whether  a technique  was 
applied  well  and  defending  the  answer),  and  Creating  (such  as  designing  a new  type  of  test.). 

For  most  of  the  material  in  the  testing  course,  we  want  students  to  be  able  to  explain  it  (conceptual 
knowledge,  remembering,  understanding),  apply  it  (procedural  knowledge,  application),  explain  why  their 
application  is  a good  illustration  of  how  this  technique  or  method  should  be  applied  (understanding,  application, 
evaluation),  and  explain  why  they  would  use  this  technique  instead  of  some  other  (analysis). 

One  of  the  key  contributions  of  taxonomies  like  Bloom’s  is  that  they  focus  us  on  assessment  techniques 
that  are  appropriate  to  the  instructional  objective. 

To  help  our  students  learn  at  a variety  of  levels,  we  had  to  support  that  learning  with  a clearly-signaled 
variety  of  assessment  techniques.  These  include  review  quizzes,  study-guide  driven  examinations,  applications  to 
the  product  under  test  through  homework  and  in-class  activities,  and  additional  classroom  activities  such  as 
orientation  exercises  and  structured  discussions. 

Review  quizzes.  Review  quizzes  are  simple  objective  tests  (such  as  multiple  choice  or  fill-in-the-blank). 

The  typical  quiz  has  ten  questions.  A student  who  watches  the  video  before  class  should  pass  each  quiz.  We  give  a 
quiz  when  we  want  to  remind  students  to  watch  the  lectures  before  coming  to  class  or  when  we  want  to  focus 
students’  attention  on  individual  details,  such  as  key  definitions.  The  quiz-review  discussion  after  the  test  creates  a 
natural  opportunity  for  discussion  of  the  details  of  interest. 

Study-guide  driven  examinations.  At  the  start  of  the  course,  we  give  students  a list  of  100  questions.  All 
midterm  and  final  exam  questions  are  drawn  from  this  pool.  We  have  sometimes  used  the  same  approach  in  the 
computer  law,  ethics  & society  course  and  the  metrics  & modeling  course.  The  Fall  2005  list  is  at 
http://www.testingeducation.org/k04/BBSTreviewfall2005.htm.  The  full  pool  that  we  draw  sets  of  100  from  is  at 
http://www.testingeducation.org/k04/EssayExam.htm  and  is  frequently  updated. 

In  previous  iterations  of  the  course,  about  half  of  the  questions  called  for  definitions,  the  other  half  for  short 
or  long  essay  questions.  We  are  now  experimenting  with  a lower  proportion  of  definitions,  relying  on  the  quizzes  to 
focus  students  on  definitions  and  other  basic  facts.  Our  choice  of  100  questions  is  based  on  student  reactions  to 
longer  and  shorter  lists.  If  the  list  seems  too  long,  students  won't  use  it,  or  not  to  the  level  that  we  want.  If  the  list  is 
too  short,  the  course  is  (in  our  view)  too  easy  and  too  narrow  (Kaner,  2003). 

We  encourage  students  to  refer  to  the  study  guide  and  work  through  the  relevant  questions  as  they  take 
each  new  section  of  the  class.  These  help  self-regulated  learners  monitor  their  progress  and  understanding — and 
seek  additional  help  as  needed.  Students  can  focus  their  studying  and  appraise  the  depth  and  quality  of  their  answers 
before  they  write  a high-stakes  exam.  Our  experience  of  our  students  is  consistent  with  Taraban,  Rynearson,  & Kerr 
(2000) — many  of  our  students  seem  not  to  be  very  effective  readers  or  studiers,  they  seem  not  to  be  very  strategic  in 
the  way  they  spend  their  study  time,  and  as  a result,  they  don't  do  as  well  on  exams  as  we  believe  they  could.  We  do 
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not  intend  to  use  this  kind  of  study  guide  in  every  course.  As  we  overhaul  more  courses,  we  will  make  strategic 
decisions  as  to  which  courses  should  offer  this  form  of  studying  assistance,  and  which  should  assume  that  students 
have  progressed  past  the  need  for  it. 

Given  that  we  use  this  approach,  we  gain  some  benefits  (Kaner,  2003): 

• This  approach  gives  students  notice  of  the  questions  and  time  to  prepare  thoughtful,  well-organized,  peer- 
reviewed  answers  to  them. 

• In  turn,  this  allows  us  to  require  well- written,  thoughtful,  well-organized  answers  on  time-limited  exams. 
This  maps  directly  to  the  testing  course's  goals  of  fostering  better  written  communication. 

• We  can  also  give  students  complex  questions  that  require  time  to  carefully  read  and  analyze,  but  that  don't 
discriminate  against  students  whose  first  language  is  not  English  because  they  have  the  questions  well  in 
advance  and  can  seek  guidance  on  the  meaning  of  a question  from  anyone  they  choose.  Additionally,  this 
creates  a cooperative  learning  task  involving  complex  concepts  for  the  whole,  in  which  these  students 
participate — this  should  help  limited-English-proficiency  students  improve  their  language  skills  (Crandall, 
1993). 

• Students  don’t  have  to  spend  time  during  the  exam  to  carefully  read  the  question,  try  to  understand  it,  and 
then  develop  an  outline  for  the  answer.  They  did  all  that  during  preparation  (or  should  have).  Therefore,  the 
instructor  can  write  an  exam  with  more  questions  to  answer  per  hour,  achieving  higher  coverage  of  the 
course  material. 

• Using  complex  questions  allows  us  to  insist  on  precise  reading.  For  example,  we  often  include  a question 
that  explicitly  states  that  the  student  should  ignore  the  possibility  of  invalid  values  when  generating  a set  of 
tests.  Those  students  who  create  tests  for  invalid  values  lose  points.  In  the  class  that  we  return  graded 
exams  and  discuss  the  grading,  we  draw  attention  to  this  error  and  remind  students  that  testers  must  be  able 
to  read  specifications  precisely,  for  example  to  note  what  is  within  the  scope  of  their  analysis  and  what  is 
not.  Similarly,  when  a question  asks  for  three  examples,  we  stop  reading  after  the  third,  and  we  say  so  in 
the  review  session.  (We  tell  students  these  things  before  the  test,  but  many  students  seem  not  to  notice  it 
until  after  the  consequence  has  happened  to  them  or  another  student  they  care  about.) 

• Students  can  study  together,  focusing  on  the  structure  provided  by  the  study  guide.  Collaborators  review 
and  critique  each  others’  answers,  improving  the  quality  of  answers  and  raising  the  level  at  which  the 
course  content  is  understood. 

We  have  been  surprised  by  recent  published  reports  that  study  guides  (or  "pedagogical  aids")  either  provide 
no  help  to  students  or  change  their  performance  by  as  little  as  two  percent.  For  example,  Dickson,  Miller,  & Devoley 
(2005)  required  students  to  use  the  study  guide  that  came  with  their  introductory  Psychology  textbook,  and  assessed 
students'  knowledge  using  a multiple-choice  exam.  The  students  who  used  the  study  guide  scored  about  2%  higher 
(a  small  but  statistically  significant  difference)  than  students  in  another  section  who  did  not  use  the  study  guide. 
Gurung  (2003,  2004)  reports  even  less  favorable  results  of  using  study  guides.  Perhaps  the  difference  is  that  their 
examinations  are  multiple  choice,  while  ours  are  essays  that  require  more  demonstration  of  cognitive  structuring  of 
the  material.  Certainly,  other  study-assisting  activities  like  concept  mapping  have  significant  positive  effects  on 
learning.  See  Zele,  Lenaerts,  & Wieme  (2004)  and  their  many  references  for  more  on  concept  mapping. 

Applications  to  the  product  under  test.  The  course  has  a designated  software  application  under  test.  The  Spring  2005 
students  tested  Open  Office's  word  processor.  Previous  semesters  focused  on  the  Open  Office  spreadsheet  or 
presentation  program,  or  on  the  Mozilla  Firefox  browser.  Each  student  joins  the  open  source  software  project. 
Applying  what  we  learn  to  a sample  application  has  been  an  important  aspect  of  the  testing  course.  In  each  of  the 
proposed  classes,  we  look  for  ways  to  credibly  tie  the  course  material  to  practical  applications.  Here  are  some  of  the 
benefits  in  the  testing  course. 

• It  makes  the  concepts  we  teach  "real"  to  the  students  by  situating  them  in  the  development  of 
products  that  are  well-known  and  well-regarded  (Lave  & Wenger,  1991). 

• Students  gain  insight  and  real-world  experience  that  they  can  (and  do)  talk  about  in  job  interviews. 

• Students  create  work  products  that  they  file  with  the  project  (such  as  bug  reports  in  the  project's 
bug  tracking  database,  or  test  cases  in  the  collection  of  test  planning  materials).  They  can  cite 
these,  show  them  off  and  explain  them  during  employment  interviews.  Feedback  from  students 
and  industry  recruiters  tell  us  several  students  effectively  use  these  work  products  in  interviews. 
Knowing  that  there  are  stakes  outside  of  the  classroom  is  a strong  motivator  for  some  students  to 
do  excellent  work. 
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• This  approach  also  facilitates  transfer  of  students'  new  knowledge  and  skills  to  the  workplace, 
because  they  are  doing  the  same  tasks  and  facing  some  of  the  same  problems  they  would  face  with 
any  commercial  software  (Clark  & Mayer,  2003). 

• When  students  create  work  products  for  a project,  we  can  take  imperfect  examples  and  rework 
them  in  class,  step  by  step,  to  show  what  a better  job  would  look  like.  In  the  most  desirable  case,  a 
few  students  submit  work  that  is  blasted  as  incompetent  by  an  irascible  programmer  or  project 
manager.  That  creates  a real-world  context  for  the  friendly,  corrective  lecture  by  the  teacher  who 
offers  coaching  on  how  to  do  something  like  this  much  better  next  time.  Worked  examples  can  be 
powerful  teaching  tools  (Clark  & Mayer,  2003)  especially  when  they  are  motivated  by  real-life 
situations. 

Students  work  with  the  product  under  test  in  several  classroom  activities  and  in  their  homework 
assignments. 

Classroom  activities. 

We  spend  classroom  contact  hours  on  coached,  small-group  activities.  For  example,  groups  of  up  to  4 
students  apply  the  lecture  material  to  a real  product  ( Open  Office)  or  discuss  a hypothetical  problem  or  work  on 
other  puzzles,  generally  problems  that  the  lecture  would  prepare  the  student  for  but  that  go  beyond  the  lecture  itself. 
We  teach  in  a lab  room,  with  one  computer  per  student.  In  class,  students  work  together  in  groups.  Activities  are 
open  book.  Students  are  encouraged  to  look  up  information  on-line.  The  instructor  moves  from  group  to  group 
asking  or  answering  questions,  giving  feedback,  or  offering  supplementary  readings  that  relate  to  the  specific 
direction  taken  by  an  individual  group. 

Only  some  of  these  activities  involve  the  application  under  test.  Others  introduce  students  to  the  material 
before  the  lecture  (preparing  them  for  the  lecture),  develop  a theoretical  point  made  in  lecture,  address  a question  in 
the  study  guide,  address  a question  raised  by  a student,  let  them  try  out  a test  tool,  or  help  the  students  work  through 
a complex  section  of  one  of  the  course  readings.  Students  often  present  results  to  the  class  in  the  last  15  minutes  of 
the  75 -minute  class.  They  often  hand  in  their  work  for  grading. 

The  diversity  of  activities  we  imagine  is  partially  inspired  by  the  diversity  of  examples  in  the  Activities 
Handbook(s)  for  the  Teaching  of  Psychology  (Benjamin  & Lowman,  1981;  Benjamin,  Nodine,  Ernst,  & Broeker, 
1999;  Makosky,  Sileo,  Whittemore,  Landry,  & Skutley,  1990;  Makosky,  Whittemore,  & Rogers,  1987).  We're  still 
learning  how  to  develop  good  activities  and  how  to  document  them.  Our  documentation  is  free-form  so  far.  Gagnon 
& Collay  (2001)  provide  what  looks  like  a good  structure  for  thinking  about  and  documenting  activities.  We  intend 
to  experiment  with  it,  and  possibly  adopt  it,  over  the  next  year. 

Orientation  exercises. 

One  interesting  type  of  activity  gives  the  students  a problem  to  work  through  that  they  probably  won't 
adequately  solve — but  that  will  be  solved  in  the  next  lecture  (Kaner,  2004).  "Cognitive  conflict  or  puzzlement  is  the 
stimulus  for  learning  and  determines  the  organization  and  nature  of  what  is  learned"  (Savery  & Duffy,  2001). 

Structured  discussions. 

In  one  of  the  in-class  activities,  students  get  a table.  The  rows  name  16  attributes  of  good  tests  (such  as 
power  of  the  test  and  credibility  of  the  test).  The  columns  list  10  common  test  techniques.  No  technique  is  good  on 
all  of  the  attributes.  For  example,  higher-power  tests  (tests  more  likely  to  expose  errors)  often  use  extreme  values 
that  are  not  representative  of  normal  use.  The  task  is  to  mark  the  three  attributes  that  a test  technique  is  designed  to 
optimize  (domain  testing  is  more  focused  on  the  power  of  tests;  scenario  testing  is  usually  more  focused  on  realistic 
uses)  and  to  mark  two  attributes  that  testers  don’t  pay  attention  to  when  they  design  a test  using  this  technique. 
Along  with  filling  out  the  chart,  the  student  writes  a paragraph  for  each  technique,  explaining  the  choices.  Students 
discuss  the  chart  in  one  class,  take  it  home  and  think  about  it,  then  discuss  it  again  in  the  next  class  and  submit  their 
work  as  individuals  or  as  a group.  This  year,  Fall  2005,  we  will  probably  spend  a third  class  day  on  this  and  ask 
students  to  resubmit  their  analysis  close  to  the  end  of  the  course. 

This  is  a difficult  but  important  assignment.  It  also  illustrates  the  use  of  an  activity  to  foster  a narrowly 
focused  discussion  among  a group  of  students. 

Drill  exercises. 

In  his  initial  work  under  NSF  Award  EIA-01 13539  ITR/SY+PE:  Improving  the  Education  of  Software 
Testers,  Kaner  expected  to  be  able  to  bring  testing  students  to  mastery  of  some  techniques  through  practice  with  a 
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broad  set  of  examples.  Padmanabhan  (2004)  applied  this  to  domain  testing  in  her  Master's  thesis  project  at  Florida 
Tech,  providing  students  with  15  classroom  hours  of  instruction,  including  extensive  practice.  All  students  were  able 
to  solve  the  problems  in  her  final  examination,  which  presented  problems  similar  to  those  solved  in  class.  However, 
when  we  added  a problem  that  was  just  slightly  more  complex  and  required  application  of  an  idea  that  had  been 
described  in  lecture  but  not  practiced,  every  student  failed  to  discover  problems  that  (in  our  view)  should  have  been 
obvious  if  the  student  understood  the  method,  rather  than  followed  a rote  procedure.  This  result  (and  some  less 
formally  conducted  prior  failures)  was  surprising  to  us,  initially  very  disappointing,  a strong  motivator  to  read 
further  in  the  STEM  education  literature  and  ultimately  redesign  the  testing  course. 

In  retrospect,  I realize  that  we  replicated  a common  finding  in  mathematics  education.  Drill  helps  people 
master  a specific  task,  but  doesn't  lead  to  generalization  and  transfer  (Bransford,  Brown,  & Cocking,  2000). 

We  still  think  that  examples  and  solved  problems  are  very  useful,  and  we  still  want  to  present  students  with 
strong  exemplars,  but  our  goal  is  to  find  ways  to  help  students  fit  them  in  a broader  context,  to  facilitate  effective 
generalization  and  successful  transfer.  Even  though  students  gain  hands-on  experience  with  the  test  techniques  (and 
the  other  subject  matter  of  the  course),  the  in-class  work  and  the  take-home  assignments  are  not  drill.  Students  are 
not  applying  a known  approach  to  several  similar  problems.  They  are  figuring  out  the  details  of  the  approach 
themselves,  applying  ideas  presented  in  lecture  and  comparing  notes. 

Take-home  assignments.  We  give  students  about  5 take-home  assignments.  They  have  a week  or  two  to 
complete  an  assignment.  It  often  comes  with  a grading  rubric.  Eventually,  all  of  them  will  come  with  rubrics  and 
some  will  come  with  samples  of  answers  to  similar  assignments.  A typical  assignment  applies  test  techniques  to  the 
product  under  test,  gathering  information  about  the  product,  its  markets  and/or  its  risks,  working  through  a standard 
or  trying  out  a tool  and  writing  an  experience  report.  Students  are  welcome  to  work  together  in  groups  of  two  or 
three. 

Notes  on  Video  Implementation 

We  are  amateurs  at  photography  and  video  production,  not  particularly  talented,  but  enthusiastic.  Our 
videos  reflect  that.  Here  are  some  good  examples  of  our  work. 

• This  one  (http://www.testingeducation.org/k04/video/FunctionTest.wmv)  on  function  testing  is  a self- 

contained  video  that  includes  a detailed  demonstration  of  the  technique. 

• These  two  (http://www.testingeducation.org/k04/video/BBSTguiAuto2.wmv  and 

http://www.testingeducation.org/k04/video/QverviewPartB.wmv)  illustrate  our  current  layout  and  pacing. 

We’ve  made  progress  in  laying  slides  out  to  be  more  readable,  in  editing  the  video  to  avoid  jarring 

transitions,  and  in  pacing  the  lecture  and  the  slide  transitions,  but  we  have  a long  way  to  go.  We  expect  to  continue 
to  make  incremental  progress  (remaining  obvious  amateurs)  in  technical  production,  but  we  hope  to  make  the 
content  more  engaging  in  a few  ways.  First,  we  expect  to  provide  more  demonstrations,  like  the  function  testing  one. 
Second,  we  want  to  add  more  engaging  stories,  like 

http://www.testingeducation.org/kQ4/video/BBSTscriptTest2.wmv.  Unfortunately,  these  take  a lot  of  work.  Third, 
we're  intrigued  by  infomercials,  which  often  heighten  interest  through  interactions  among  several  actors,  including 
scripted  discussions  that  pose  questions  to  address  specific,  predictable  areas  of  confusion  or  resistance  to  a new 
idea. 

We  record  the  videos  at  home  using  a mid-priced  digital  camcorder  (a  Sony  DCR  TVR-38).  We  download 
the  videos  to  a Dell  XPS  Gen5  computer  with  a dual-core  Pentium  chip  with  hyperthreading.  Processor  speed  and 
the  ability  to  do  multiprocessing  are  important  for  editing  and  rendering  (saving  the  edited  video  in  a final  format,  in 
our  case  QuickTime  or  Windows  Media  Video).  A video  that  renders  in  an  hour  (on  the  Gen5)  rendered  in  up  to  8 
hours  on  slower  machines.  When  an  error,  such  as  corrupted  input,  a memory  leak  or  other  bug,  halts  the  rendering 
process,  rendering  seems  to  either  fail  right  away  or  near  the  end  of  processing.  On  the  predecessor  to  the  Gen5  (an 
Alienware  Roswell  with  a Matrox  video  coprocessor),  failures  were  common — redoing  the  rendering  could  add  30 
hours  of  rework  for  a 45 -minute  lecture  segment.  Producing  videos  on  schedule,  on  time  for  students  to  be  able  to 
watch  them  before  the  scheduled  class,  is  much  more  challenging  the  slower  the  rendering  process  and  the  less 
reliable  the  computer.  We  use  the  Windows  systems  in  order  to  use  Adobe  Premiere  Pro.  Both  of  us  have  our  own 
Macintoshes  and  Windows  systems  (and  have  for  years).  We  prefer  the  Macs,  but  our  pilot  studies  (two  full  lectures 
and  some  other  fooling  around)  convinced  us  that  Premiere  was  more  reliable — at  least  in  the  way  we  worked  with 
video  editing — than  Final  Cut  Pro , and  Premiere  isn’t  being  updated  for  the  Mac.  Looking  at  current  prices  for  our 
setup,  it  costs  about  $8000  for  a fast  and  reliable  computer  with  lots  of  memory,  three  hard  disks  (source  data, 
resulting  video,  other  data  and  work  area),  Premiere , and  some  utilities  like  Snaglt  (for  screen  shots  and  videos  of 
running  programs). 

Creating  a lecture  includes  preparation  tasks  as  well  as  taping  and  editing  a video.  These  include  polishing 
the  slides  to  get  the  flow  of  the  lecture  right,  (very  imperfectly)  balance  the  time  to  be  spent  per  slide,  laying  out  the 
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slides  to  be  readable  after  video  rendering  in  a compressed  format,  writing  out  detailed  notes  for  the  lecture  and 
rehearsing  the  lecture  before  taping. 

Giving  the  lecture,  on  tape,  also  takes  time.  We  might  tape  the  presentation  of  a single  slide  several  times 
until  it  finally  feels  right,  then  move  to  the  next  slide.  An  hour  of  final  tape  might  take  3 to  10  hours  of  videotaping, 
followed  by  4.5  to  15  hours  (about  1.5  hours  per  hour  of  tape)  of  editing. 

From  “start”  to  finish,  where  “start”  means  we  already  have  good  slides  and  a fairly  clear  idea  of  the  lecture 
(with  lecture  experience  with  these  slides),  lecture  material  to  support  one  classroom  contact  hour  (perhaps  30-50 
minutes  of  video)  took  about  35  hours  last  year  (the  unreliable  and  slightly  slower  Alienware  system).  On  the  newer 
system,  we  hope  to  stabilize  at  25  hours  video  preparation  time  per  class. 

Along  with  video  preparation  is  preparation  of  all  of  the  other  supporting  materials.  Freeing  up  all  of  the 
classroom  contact  hours  for  labs  and  seminars  means  that  we  have  to  create  laboratory  and  seminar  activities, 
develop  ways  to  evaluate  them,  and  provide  feedback  to  the  students. 

The  overall  preparation  time  cost,  borne  primarily  by  the  faculty  member,  is  enormous  and  far  beyond  what 
we  suspect  any  university  would  pay  for.  It  is  a labor  of  love.  For  us,  the  driving  passion  is  for  improvement  of  the 
state  of  the  practice  in  the  broad  field  of  software  testing.  In  a field  that  is  educationally  underserved  (academically 
and  commercially),  providing  high-quality  free  course  materials  to  the  community  at  large  might  have  an  impact  on 
the  development  of  basic  knowledge  and  skill  in  the  field. 

To  enable  this  broad  distribution,  we  have  to  keep  the  intellectual  property  rights  to  our  course.  That  makes 
it  possible  for  us  to  license  the  material  any  way  we  want,  including  Creative  Commons.  Some  schools  (K-12  and 
university)  have  begun  asserting  ownership  of  copyrightable  teaching  materials  created  by  their  faculty.  In  these 
cases,  the  school  might  not  approve  of  publishing  courseware  that  they  own,  for  free. 

We  planned  for  this  by  setting  up  to  do  all  of  the  work  at  home,  using  our  own  equipment,  and  so  far,  we 
have  done  all  of  the  videotaping  and  editing  at  home.  Eventually,  Florida  Institute  of  Technology  approved  a stored 
course  policy  that  allows  us  to  keep  ownership  of  materials  we  create  at  school  so  long  as  we  don’t  rely  heavily  on 
school  resources  (The  final  policy  is  almost  the  same  as  the  proposal  we  reported  in  Kaner,  2002).  This  has  allowed 
us  to  supplement  our  materials  with  work  done  by  research  assistants  in  Kaner’ s lab  at  Florida  Tech  and  will  allow 
us  to  do  more  work  at  school  and  more  in  collaboration  with  other  faculty,  in  the  future. 

A Broader  Market 

The  course  appears  to  be  working  well  in  our  classes  at  Florida  Tech.  We  are  still  evaluating  comparable 
performances  (such  as  final  exams  from  previous  teachings  of  the  course,  compared  to  those  from  the  new 
approach).  Our  impression  from  informal  comparison  is  that  the  students  are  generating  better  work.  We  are  also 
getting  favorable  student  evaluations. 

In  addition  to  local  teaching,  we  make  the  course  available  on  the  Web,  for  free.  About  120  people  have 
signed  up  for  a low-traffic,  moderated  email  list  for  people  who  teach  a testing  course  using  our  materials.  Some 
teach  it  at  university,  others  at  their  own  company.  Some  are  self-studiers  trying  to  gain  insight  from  discussions 
among  the  teachers.  The  list  has  been  in  operation  for  about  six  months,  but  we  are  still  just  getting  started  on 
serious  discussions.  Eventually,  we  expect  feedback  from  this  list  and  from  a second  list  (discussion  by  students)  to 
provide  a steady  stream  of  insights  into  the  weaknesses  of  the  course. 

Informal  feedback  so  far  is  positive  and  enthusiastic,  but  it  is  clear  that  these  materials  are  far  from  ideal  for 
self-study  and  that  several  instructors  (especially  commercial  trainers)  could  use  more  help  on  evaluating  the  results 
of  tests  and  assignments. 

Closing  Notes 

This  has  been  a time-consuming  course  to  create,  and  we’re  not  done.  We  have  more  lectures  to  tape,  plus 
more  grading  samples,  examples,  and  other  instructional  supports. 

We  see  the  work  as  worth  the  effort,  partially  because  it  appears  to  be  improving  our  on-campus  students’ 
learning  and  enthusiasm,  but  especially  because  it  is  being  reused  around  the  world  and  will  be  used  even  more 
broadly  as  it  gets  more  polished.  We  also  expect  the  availability  of  a good  free  course  to  stimulate  improvement  in 
the  leading  commercial  courses,  in  order  to  compete  well  with  our  free  offering.  Some  commercial  instructors  are 
already  incorporating  some  of  our  materials  in  their  courses  (which  they  are  welcome  to  do  under  our  Creative 
Commons  license).  Others,  we  are  told,  have  started  their  upgrading  process  without  using  our  material. 

We  see  this  work  as  promising  enough  that  we  are  already  planning  the  next  few  courses.  Eventually,  we 
hope  to  offer  an  open  source  curriculum  in  software  testing,  rather  than  just  one  or  two  courses.  As  software 
development  work  disperses  ever  more  around  the  world,  a set  of  high  quality  courses  available  to  everyone  for  free 
offers  the  possibility  of  a reasonably  high  baseline  level  of  knowledge  of  the  field  everywhere — or  as  competition 
develops,  of  several  good  baselines  that  present  competing  visions  of  the  field  and  its  operations. 
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ENVIRONMENT  FOR  VETERINARY  MEDICINE  STUDENTS 
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Abstract 

Veterinary  Medicine  Diagnostic  Tool  (VMTD)  is  a prototype  constructivist  learning  environment.  Four 
among  six  components  of  Constructivist  Learning  Environments  (Jonassen,  1999)  were  considered  in  designing  the 
prototype  ofVMDT.  VMTD  centers  cases  to  drive  students  to  acquire  knowledge  and  skills  by  solving  problem 
cases,  provides  related  cases  to  increase  cognitive  flexibility,  includes  support  tools  such  as  question  editors,  and 
contains  resources  that  are  useful  for  solving  problems. 

Introduction 

Diagnostic  reasoning  is  among  the  most  important  problem  solving  skills  that  veterinary  students  need  to  learn 
because  the  primary  work  of  professional  veterinarians  is  to  diagnose  patients.  However,  veterinary  students  often 
find  it  very  difficult  to  acquire  this  requisite  skill.  Before  students  come  to  the  clinical  pathology  classroom,  they  are 
often  taught  to  memorize  factual  knowledge  and  medical  terms  in  anatomy,  physiology  and  other  relevant  areas. 
After  this,  they  are  suddenly  required  to  diagnose  cases  by  applying  many  varying  types  of  knowledge  in  order  to 
interpret  lab  data  in  the  clinical  pathology  classroom.  Although  case-based  instruction  provides  veterinary  students 
opportunities  for  experiences  that  are  similar  to  those  of  practicing  veterinarians,  this  sudden  change  may  cause 
significant  frustration  and  difficulty  for  students,  as  they  have  not  been  previously  exposed  to  such  knowledge 
application.  Case  diagnosis  is  very  complex  and  requires  a large  amount  of  domain  knowledge,  strategic  knowledge, 
and  performance  knowledge.  In  order  to  support  the  veterinary  students’  construction  of  these  types  of  knowledge 
and  skills  in  a more  effective  manner,  we  designed  the  Veterinary  Medicine  Diagnostic  Tool  (Figure  1).  In  the 
present  paper  we  describe  professional  veterinarians’  diagnostic  reasoning  process,  the  feature  of  the  system,  and 
how  it  supports  knowledge  construction  of  diagnostic  reasoning. 

Task  Analysis  of  Diagnostic  Reasoning 

In  order  to  analyze  professional  veterinarians’  cognitive  behavior  in  case  diagnosis,  a series  of  interviews  with 
two  professional  veterinarians  was  conducted  using  the  PARI  method,  a type  of  cognitive  task  analysis  (Hall,  Gott, 
& Pokorny,  1995).  PARI  is  an  acronym  of  Precursor,  Action,  Result,  and  Interpretation.  Although  a complete 
description  of  the  PARI  model  is  beyond  the  scope  of  this  abstract,  a more  detailed  explanation  can  be  found  in  the 
book  of  Task  Analysis  Methods  for  Instructional  Design  (Jonassen,  Tessmer,  & Hannum,  1999).  By  posing 
questions  to  two  professional  veterinarians  and  asking  them  to  “think  aloud”,  we  were  able  to  identify  the  domain 
knowledge,  strategic  knowledge,  and  performance  knowledge  needed  to  successfully  diagnose  a case.  Veterinarians 
first  come  up  with  a differential  diagnosis  based  on  an  interpretation  of  the  initial  state  of  the  patient  and  generate 
various  questions  that  need  to  be  investigated  and  verified.  It  is  after  this  differential  diagnosis  that  the  veterinarians 
order  the  lab  tests  considering  the  cost  associated  with  them,  interpret  the  lab  data,  and  come  to  the  final  diagnosis. 

Theoretical  Basis  of  the  VMDT  Design 

VMDT  is  designed  based  on  the  model  for  designing  Constructivist  Learning  Environments  (CLEs)  that 
consists  of  a problem  and  “intellectual  support  systems”  (Jonassen,  1999).  The  six  essential  components  in  the 
model  for  designing  CLEs  are  as  follows. 

■ Question/Case/Problem/Project 

■ Related  Cases 

■ Information  Resources 

■ Support  Tools 

■ Conversation  and  Collaboration  Tools 

■ Social/Contextual  Support 
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Four  among  six  components  of  CLEs  were  considered  in  designing  the  prototype  of  VMDT.  In  the  model  for 
CLEs,  the  problem  is  the  center  of  the  environment  and  drives  students  to  acquire  the  requisite  domain  knowledge  to 
solve  the  problem.  The  problems  in  CLEs  should  be  what  practitioners  do  in  real  work  settings  and  should  require 
students  to  engage  in  authentic  cognitive  challenges  that  are  similar  to  those  of  practitioners  in  the  real  world 
(Jonassen,  1999).  However,  students  lack  many  of  the  experiences  that  are  very  important  in  solving  problems. 
Thus,  the  model  for  CLEs  requires  a type  of  case  library  of  related  cases  in  order  to  scaffold  student  memory  with 
represented  experiences  and  to  enhance  the  complexity  of  cases  by  providing  multiple  perspectives.  In  addition  to 
the  case  library,  essential  parts  of  CLEs  are  information  resources  that  are  important  and  necessary  for  solving  the 
problem  and  tools  that  support  student  problem  solving  and  dynamic  activities. 

Vet  Med  Diagnostic  Tool 

The  Diagnostic  Tool  is  a prototype  for  a case-based  learning  environment  that  requires  veterinary  students  to 
diagnose  cases  in  a manner  that  is  analogue  to  practitioners.  VMDT  consists  of  authentic  cases  that  the  professional 
veterinarians  encounter  in  their  work  settings.  After  receiving  the  case  about  the  animal’s  wellness  and  symptoms, 
the  learner  starts  building  his/her  differential  diagnosis  by  selecting  possible  diagnoses  provided  in  the  Differential 
Diagnosis  Editor.  Then,  the  learner  is  required  to  generate  questions  that  need  to  be  asked  to  validate  each  diagnosis. 
This  action  is  the  same  as  the  veterinarians’  initial  diagnosis  process.  In  real  work  settings,  the  veterinarian  will  ask 
implicit  questions  to  him/herself  to  verify  his/her  initial  diagnosis  before  ordering  lab  tests  to  run.  Then,  the  learner 
decides  what  lab  tests  he/she  wants  to  run  based  on  their  questions.  Once  the  learner  orders  the  lab  tests,  the  patient 
data  will  appear  in  the  screen  along  with  slides.  In  reality,  there  are  many  constraints  such  as  costs  in  ordering  lab 
tests.  The  Vet  Med  Diagnostic  Tool  has  a built-in  cash  register  to  track  the  cost  of  each  test.  After  the  learner 
finishes  reviewing  the  lab  data,  he/she  will  finalize  his/her  diagnosis  in  the  “Your  Diagnosis”  area  and  is  then 
required  to  provide  justification  and  reasoning  behind  their  final  diagnosis.  At  each  stage  of  working  through  the 
problem,  the  learner  has  opportunities  to  save  their  work  and  return  to  the  point  where  he/she  left  off.  Also,  helpful 
information  such  as  relevant  articles  can  be  accessed. 

Figure  1.  Vet  Med  Diagnostic  Tool 


Vet  Med  Prototype  Diagnostic  Tool 

Case  History  Differential  Diagnosis  Your  Quesbons  Paired  Dala  Ydut  Diagnoas 


Differential  Diagnosis 


1 Patient  has  anemiaQ 

2 Patient  has  kidney  disease  Q 

3.  Patient  has  acute  pancreatitis  Q 


Vet  Med  Prototype  Diagnostic  Tool 

Case  history  Differential  Diagnosis  Your  Quesbons  P&1»ent  Dala  Your  Diagno»s 


Available  Patient  Data 


The  Vet  Med  Diagnostic  Tool  supports  veterinary  students  construct  their  knowledge  and  skills  for  diagnostic 
reasoning  through  a real  case  diagnosis.  They  are  engaged  in  the  thinking  process  like  practitioners  in  real  working 
settings  and  must  provide  the  reasons  for  their  actions  and  diagnosis. 
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Introduce  a Compressor/Decompressor  Technology  to  Enhance  the 

Multimedia  Presentations 

Daesang  Kim 
Pomsook  Tantrarungroj 
Indiana  State  University,  Indiana 

Abstract 

This  paper  introduces  information  about  using  codec  technology  in  order  to  produce  an  appropriate  video 
presentation.  The  information  from  the  study  serves  as  a tool  for  users  to  choose  the  appropriate  technology  for 
their  video  presentations.  Several  comparison  tables  are  provided  in  order  to  help  users  to  choose  an  appropriate 
technology  for  their  presentations.  In  general,  we  found  that  the  bigger  size  files  have  less  compression  and  the 
smaller  size  files  are  more  compressed.  Also,  the  bigger  size  files  have  higher  quality  than  the  smaller  size  files. 

Introduction 

With  more  fast  and  large  hard  disk  computers  and  powerful  multimedia  programs  widely  available, 
teachers  and  students  are  able  to  use  multimedia  presentations  with  the  help  of  computers  more  efficiently  and 
effectively.  Multimedia  consists  of  more  than  one  combination  of  text,  audio,  graphic,  animation,  and  video.  Using 
multimedia  presentations  by  teachers  will  hold  students’  interest  in  the  classroom.  Many  educators  believe  that  using 
multimedia  is  essential  when  working  with  today’s  video  generation  (Jonassen,  2000).  Video  presentations  can  help 
students  in  a visual  way  that  reinforces  what  verbal  or  static  graphic  alone  cannot.  With  the  available  storage  and 
delivery  technology,  presenters  can  choose  various  delivery  technologies  such  as  a hard-disk,  a floppy-disk,  a zip- 
disk,  a CD,  a DVD,  and  the  Internet.  However,  constructing  a computer-based  video  in  multimedia  presentations 
more  likely  requires  technology  like  codec. 

Why  do  we  need  a video  codec? 

To  create  a Digital  Video  (DV),  use  of  an  appropriate  codec  is  necessary.  Otherwise,  the  file  size  of  the 
DV  will  be  very  large.  The  common  types  of  DV  camcorders  that  are  used  for  storing  video  on  camcorder  are 
Digital8,  MiniDV,  MicroMV,  and  DVD.  DV  consumes  a lot  of  hard  disk  space;  for  instance,  a MiniDV  tape 
consumes  about  215MB  of  disk  space  per  minute,  or  about  13GB  per  hour  (Johnson,  Stauffer,  & Broida,  2004). 

This  means  that  the  user  must  have  enough  available  space  on  a computer  to  transfer  the  video  data  from  a DV 
camcorder.  An  IEEE  1394  (also  known  as  FireWire  and  in  Sony  parlance,  i.LINK)  interface  helps  teachers  transfer 
video  data  from  a DV  camcorder  to  a computer.  For  instance,  a teacher  may  easily  transfer  one  hour  of  video  data 
from  a DV  camcorder  to  a computer  through  an  IEEE  1394  interface.  And  then,  the  teacher  may  simply  present  the 
1 hour  video  from  the  hard  drive.  However,  if  the  teacher  has  to  move  to  another  classroom,  it  would  be  difficult  for 
the  teacher  to  bring  the  entire  computer  for  the  presentation.  Codec  technology  will  solve  this  problem.  It  helps  to 
reduce  the  file  size  without  noticeable  loss  of  quality  to  distribute  the  video  to  CD  (650MB),  DVD  (4.7GB),  and  the 
Internet. 

What  is  a codec  technology? 

Codec  is  a short  form  of  compressor/decompressor  that  manages  compression.  The  goal  of  video 
compression  is  to  represent  the  same  content  using  less  data  (Adobe  Premiere  Pro:  Classroom  in  a book,  2004).  It 
has  various  types  of  computer  algorithms  that  compress  and  decompress  audio,  video,  and  image  files.  All  codecs 
can  be  divided  into  two  categories:  lossless  and  lossy.  Lossless  compression  means  that  we  do  not  lose  any  data 
when  we  compress  or  decompress  the  video.  On  the  other  hand,  with  lossy  compression,  when  we  compress  a video 
and  then  decompress  it,  we  do  not  retrieve  the  original  quality  of  the  video  (Currier,  1995).  In  general,  codecs  are 
used  to  decrease  a file  size  without  noticeable  loss  of  quality.  For  instance,  in  our  experiment  an  MPEG  compresses 
a video  file  to  a much  smaller  size  with  a ratio  of  9:1  or  more  without  a noticeable  loss  of  quality. 

Unfortunately,  to  open  and  watch  the  compressed  video,  the  same  codec  must  be  used  to  decompress  the 
video.  If  the  end-users’  computer  does  not  have  the  appropriate  codec  to  decompress  the  video,  they  will  be  unable 
to  open  or  watch  the  compressed  video  clearly.  Thus,  to  select  an  appropriate  codec,  a presentation  developer 
should  be  able  to  use  the  most  common  codec  for  end-users’  computers,  or  at  least  provide  a codec  which  is  used  by 
the  developer. 
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What  makes  an  appropriate  codec? 

How  do  we  select  an  appropriate  codec  from  the  available  codecs?  Choosing  an  appropriate  codec  for 
multimedia  presentations  can  be  a difficult  task.  In  general,  presenters  might  want  both  a small  file  size  and  high 
quality.  Buehler  (2003)  focused  on  four  areas  on  which  a codec  can  be  judged:  image  quality,  file  size, 
compatibility,  and  usability.  In  this  paper,  we  have  focused  on  three  factors:  image  quality,  file  size,  and 
compatibility,  for  selecting  an  appropriate  codec  for  multimedia  presentations. 

• Image  Quality:  Presenters  do  not  want  to  lose  their  quality  of  video  unnecessarily.  They  probably 
want  to  show  their  video  without  noticeable  loss  of  quality  for  the  audience.  They  want  to  keep  as 
good  quality  as  they  possibly  can.  In  general,  image  quality  does  significantly  affect  the  file  size.  If 
they  adjust  an  image  quality  simply  to  produce  a small  file  size  video  without  a good  codec,  it  will 
reduce  a file  size,  but  presenters  will  lose  quality  of  video  unnecessarily. 

■ File  Size:  Presenters  do  not  want  to  produce  a large  file  size  video  unnecessarily.  They  do  not  want  to 
produce  a huge  video  like  2GB  for  a lOmin  video  presentation.  Depending  on  storage  like  Zip-disk 
(100MB),  hard-disk,  CD-Rom  (650MB),  DVD-Rom  (4.7GB),  and  WWW,  they  absolutely  need  a 
reasonable  file  size  that  can  be  presented.  It  is  extremely  difficult  to  deliver  a 2GB  video  file  on  the  Web. 

■ Compatibility:  Presenters  want  to  produce  a compatible  video  to  watch  easily  for  end-users.  They  do  not 
want  to  present  a video  which  requires  hardware  or  software  to  watch  it.  If  presenters  have  a good  codec  to 
preserve  good  quality  with  extremely  reduced  file  size,  but  if  end-users  can’t  watch  it  without  purchasing  a 
specific  hardware  or  software,  is  it  a good  codec? 

Methodology 

For  our  experiment,  we  created  one  short  video.  The  sequence  consisted  of  live  scenes  of  flowers  swaying 
in  the  wind,  and  was  taken  by  a Sony  Digital  Handycam  (DCR-TR  V33).  For  capturing  the  video  sequence  from  the 
DV  Camcorder  to  the  computer,  we  used  Adobe  Premiere  with  IEEE  1394  (FireWire)  interface.  The  capture 
settings  were  as  below: 

■ Capture  format:  DV/IEEE  1394  Capture 
Compressor:  Microsoft  DV  (NTSC) 

Frame  size:  720  * 480 

Frame  Rate:  29.97  FPS 
Length:  5 seconds 
Original  file  size:  18,700  KB 

Procedure 

Different  codec  technology  in  the  Adobe  Premiere  6.0  and  Premiere  Pro  was  applied  on  the  video  sequence 
in  order  to  present  the  useful  information  of  codecs  with  the  three  factors.  In  Adobe  Premiere  6.0  and  Premiere  Pro 
the  following  file  formats  are  available:  Microsoft  AVI  and  DV  AVI,  Animated  GIF,  MPEG  Real  Media, 
QuickTime,  and  Windows  Media  (See  Figure  1). 


Figure  1.  The  Export  Movie  Setting  for  selecting  a file  type  in  Adobe  Premiere  6.0. 
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We  focused  on  three  video  file  formats:  QuickTime,  AVI,  and  MPEG,  for  evaluating  variable  codecs. 

■ QuickTime:  QuickTime  was  originally  developed  for  the  Mac,  but  it  has  been  widely  used  on  the  PC.  It 
has  a MOV  file  extension  and  is  one  of  the  oldest  and  the  commonly  used  for  Web-based  content.  An  end- 
user  might  have  to  install  QuickTime  player  to  view  a video  because  it  doesn’t  come  installed  with 
Windows. 

■ Microsoft  AVI  (Audio  Video  Interleaved):  AVI  is  defined  by  Microsoft.  It  has  an  AVI  file  extension  and  is 
the  most  common  format  for  video  on  the  PC.  An  end-use  might  have  to  update  new  Windows  Media 
Player  to  view  a video. 

■ MPEG  (Moving  Picture  Experts  Group):  MPEG  is  one  of  the  oldest  compressed  video  formats  in  the 
business.  It  is  commonly  used  on  the  WWW  and  in  PC -based  video.  DVDs  and  digital  camcorders  both 
use  MPEG  compression.  (Johnson,  Stauffer,  and  Broida,  2004). 

Then  we  used  each  file  type  with  variable  codecs  for  our  experimentation  (See  Figure  2). 


Figure  2.  The  Export  Movie  Setting  for  selecting  a compressor  in  Adobe  Premiere  6.0. 

To  use  variable  codecs,  we  used  two  different  computers;  one  does  not  have  a hardware  capture  board  with  Adobe 
Premiere  Pro  (See  Table  1),  the  other  has  a hardware  capture  board,  Matrox  RT  2500  with  Adobe  Premiere  6.0  (See 
Table  2).  We  used  the  frame  size,  720  * 480  as  capturing  frame  size  of  original  video  for  producing  final  video. 

Finally,  we  investigated  WMV  and  Real  Media  file  formats  (See  Table  3)  with  different  settings  in  Adobe 
Premiere  Pro  (See  Figure  3)  for  comparing  distribution  over  the  WWW. 

■ WMV  (Windows  Media  Video):  WMV  is  Microsoft’s  highly  compressed  video  format  and  it  has  a WMV 
file  extension.  WMV  is  one  of  the  latest,  but  it  quickly  spreads  across  the  WWW.  An  end-user  might  have 
to  update  new  Windows  Media  Player  to  view  a video. 

■ Real  Media  (RM):  Real  Media  is  a proprietary  format  owned  by  Real  Networks.  It  has  a RM  file  extension 
and  is  used  for  distributing  video  over  the  WWW.  An  end-user  might  have  to  install  RealPlayer  to  view  a 
video  because  it  doesn’t  come  installed  with  Windows. 
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Transcode  Settings 


Format: 


Windows  Media 


Preset: 

Windows  Media  Video  S for  Dial-up  Modems  (5G  Kbps)  v 

WM8  PAL  download  1 024 

Comment: 

WMSPAL  download  12S 

WM8PAL  download  25G 

WM8PAL  download  32 

Summar 

WMSPAL  download  512 

General 

WM8  PAL  download  G4 

W Video 

WM9  NTSC  1024K  download 

WM9  NTSC  128K  download 

P Audio 

WM9  NTSC  25GK  download 

” Metadata 

WM9NTSC32K  download 

Add/F 

WM9NTSC512K  download 

Clear 

WM9NTSCG4K  download 

WM9  NTSC  streaming  modem 

” Audience 

WM9  NTSC  streaming 

Add/I 

WM9PAL  download  1024 

Targe 

WM9PAL  download  128 

WM9PAL  download  256 

WMSPAL  download  32 

WM9PAL  download  512 

WM9  PAL  download  G4 

Figure  3.  The  Transcode  Setting  for  selecting  Preset  in  Adobe  Premiere  Pro. 


Ratings 


The  measurements  of  the  codecs  with  three  factors,  image  quality,  file  size,  and  compatibility,  were  as 

below. 

■ Image  Quality:  To  judge  image  quality  is  a very  difficult  task,  so  we  decided  to  use  the  frame  size,  720  * 
480,  to  judge  the  video  for  Table  1 and  2.  But  for  Table  3,  we  used  the  frame  size,  320*240,  to  judge  the 
image  quality  over  the  WWW.  We  used  three  ratings  for  measuring  this  factor,  Excellent,  Good,  and  Fair. 
The  images  below  show  how  we  judged  the  video. 


- Excellent: 
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■ File  size:  We  showed  the  file  size  in  KB  (Data  storage  capacity). 

■ Compatibility: 

- Yes  (Watchable  video  with  QuickTime  Player  5.0.2,  Real  Media  Player  9,  or  Windows  Media  Player  9.0) 

- No  (Require  to  install  additional  software  to  decompress  or  purchase  hardware) 

Result 

We  provide  three  tables  to  summarize  our  investigation.  Table  1 showed  the  results  of  using  Adobe 
Premiere  Pro  without  a hardware  capture  board.  Table  2 showed  the  results  of  using  Adobe  Premiere  6.0  with  a 
hardware  capture  board,  Matrox  RT  2500. 

Table  1 Compare  codecs  without  a hardware  capture  hoard 


Compressor 

Quality 

Size 

Compatibility 

Cinepak 

Excellent 

13,717  KB 

Yes 

S 

H.261 

Excellent 

10,572  KB 

Yes 

g 

AA 

H.263 

Excellent 

6,461  KB 

Yes 

a 

Sorenson  Video 

Excellent 

17,299  KB 

Yes 

£h 

Cinepak  Codec  by  Radius 

Excellent 

14,982  KB 

Yes 

2 > 

Indeo  Video  5.10 

Excellent 

18,357  KB 

Yes 

O < 

s 

Indeo  Video  4.5 

Excellent 

20,236  KB 

Yes 

MPEG-1 

Good 

1,259  KB 

Yes 

MPEG- 1 -VCD 

Fair 

915  KB 

Yes 

MPEG2 

Excellent 

Video  3,743  KB 

Yes 

a 

s 

Ph 

Audio  140  KB 

2 

MPEG2-DVD  Video 

Excellent 

Video  2,498  KB 

Yes 

Audio  958  KB 

MPEG2-SVCD 

Excellent 

1,705  KB 

Yes 

Note : The  table  did  not  include  several  codecs  which  had  larger  than  original  file  size  as  below. 

- QuickTime:  Component  Video,  H.263,  Intel  Indeo  Video  4.4  , Motion  JPEG  A,  Motion  JPEG  B,  MPEG-4 
Video,  Photo- JPEG,  Planar  RGB,  PNG,  Sorenson  Video  3,  and  Video. 

- AVI:  Microsoft  RLE  and  Microsoft  Video  1. 


Table  2 Compare  codecs  with  a hardware  capture  board,  Matrox  RT  2500 


Compressor 

Quality 

Size 

Compatibility 

Cinepak 

Excellent 

13,668  KB 

Yes 

S 

g 

H.261 

Excellent 

9,841  KB 

Yes 

AA 

H.263 

Excellent 

6,107  KB 

Yes 

a 

Sorenson  Video 

Excellent 

15,216  KB 

Yes 

4h 

Cinepak  Codec  by  Radius 

Excellent 

15,005  KB 

Yes 

icrosc 

AVI 

Indeo  Video  5.10 

Excellent 

16,676  KB 

Yes 

S 

Matrox  MPEG-2  I frame 

Excellent 

13,439  KB 

Yes 

a 

w 

LSX  MPEG-1 

Excellent 

3,685  KB 

Yes 

Ph 

S 

LSX  MPEG-2 

Excellent 

3,747  KB 

Yes 
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Note : The  table  did  not  include  several  codecs  which  had  larger  than  original  file  size  as  below. 

- QuickTime:  Animation,  BMP,  Component  Video,  DV  NTSC,  Graphic,  Intel  Indeo  Video  4.4,  Motion  JPEG  A, 
Motion  JPEG  B,  Photo- JPEG,  Planar  RGB,  PNG,  TGA,  TIFF,  and  Video. 

- AVI:  LEAD  MCMP  MJPEG,  Microsoft  RLE  and  Microsoft  Video  1. 


In  addition,  we  investigated  Window  Media  Video  (WMV)  and  Real  Media  file  types  with  different 
settings  for  comparing  distribution  over  the  Internet.  Table  3 showed  the  results  of  using  Adobe  Premiere  Pro 
without  a hardware  capture  board. 


Table  3 WMV  and  QuickTime  file  formats  with  different  settings  in  Adobe  Premiere  Pro. 


> 

§ 

£ 


£ S 
Pi  ^ 


Preset 

Compressor 

Quality 

Size 

Compatibility 

NTSC  64K  download 

WMV9 

Fair 

66  KB 

Yes 

NTSC  256K  download 

WMV9 

Good 

152  KB 

Yes 

NTSC  1024K  download 

WMV9 

Excellent 

800  KB 

Yes 

NTSC  Streaming  Modem 

WMV9 

Good 

448  KB 

Yes 

NTSC  Streaming  Broadband 

WMV9 

Excellent 

2,556  KB 

Yes 

NTSC  64K  download 

RM9 

Fair 

60  KB 

Yes 

NTSC  256K  download 

RM9 

Good 

149  KB 

Yes 

NTSC  1024K  download 

RM9 

Excellent 

633  KB 

Yes 

NTSC  Streaming  Modem 

RM9 

Fair 

179  KB 

Yes 

NTSC  streaming  Broadband 

RM9 

Excellent 

1,477  KB 

Yes 

Conclusion 

In  general,  we  found  that  the  bigger  size  files  have  less  compression  and  the  smaller  size  files  have  more 
compression.  Also,  the  bigger  size  files  have  higher  quality  than  small  size  files.  The  summary  of  the  study  is 
shown  in  the  following  list: 

■ H.263  codec  was  the  best  for  QuickTime  file  format  with  a sufficiently  decreased  file  size  without 
noticeable  loss  of  quality. 

■ Cinepak  Codec  by  Radius  for  AVI  file  format  slightly  decreased  file  size  without  noticeable  loss  of  quality. 
However,  if  presenters  have  a hardware  capture  board  like  Matrox  RT  2500,  Matrox  MPEG-2  I frame 
codec  would  be  an  additional  option  for  AVI  file  format. 

■ MPEG-2  codec  of  MPEG  file  format  significantly  decreased  the  file  size  without  noticeable  loss  of  quality. 
As  for  DVD-Rom  usage,  MPEG-2  was  better  because  of  its  excellent  quality  even  though  its  size  was  not 
decreased  as  MPEG-1. 

■ Several  codecs,  which  had  larger  file  size  than  the  original  file  size,  might  require  purchasing  hardware  or 
software  for  encoding  video.  We  did  not  provide  the  information  about  those  codecs. 

■ In  general,  both  WMV9  and  RM9  codecs  for  delivering  video  on  the  WWW  reduced  file  size  and  yet 
remained  good  quality. 
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Using  a concept  map  as  a tool  in  vocabulary  learning:  a Web-based  self- 
instruction  program  for  Limited  English  Proficiency  (LEP)  students 

Daesang  Kim 
Indiana  State  University 

Abstract 

This  paper  introduces  a Web-hased  self-instruction  program  that  utilizes  concept  maps  to  teach  vocabulary 
to  Limited  English  Proficiency  (LEP)  students.  Based  on  the  assumption  that  concept  maps  are  an  effective  method 
of  organizing  and  introducing  English  vocabulary,  this  program  is  designed  to  assist  students  in  conceptualizing 
and  internalizing  words.  This  instructional  program  consists  of  combinations  of  text,  audio,  graphics,  animation, 
and  video  and  can  be  accessed  on  the  Web. 


Introduction 

Designing  new  learning  tools  using  technologies  can  be  a real  challenge  for  teachers  and  instructional 
designers.  Advanced  computer  technology  helps  educators  to  design  learning  environments  that  involve  more 
complex  interactions  between  learners  and  instructional  content  (Reiser,  2001).  With  computer  technology,  the  Web 
has  become  a common  choice  in  higher  education  institutions  (Anderson,  Bauer,  & Speck,  2002).  The  various 
media,  such  as  text,  graphics,  audio,  and  video  for  delivering  content,  has  attracted  many  students  to  use  the  Internet 
for  distance  education  (Ali,  2003). 

Recently  students  who  have  limited  English  proficiency  (LEP)  are  currently  the  fastest-growing  population 
in  public  schools  in  the  United  States.  According  to  Smith-Davis  (2004), 

Between  the  years  1991  and  1999,  the  number  of  language -minority  school-aged  children  who  are  now 
living  in  the  United  States  rose  from  8 million  to  15  million,  and  the  number  ofK-12  students  who  are 
classified  as  being  limited-English-proficient  (LEP)  increased  from  5.3  million  to  10  million  students. 
Despite  the  fact  that  only  eight  languages-Spanish,  Vietnamese,  Hmong,  Cantonese,  Cambodian,  Korean, 
Laotian,  and  Navajo-comprise  85%  of  linguistic  diversity,  350  language  groups  are  actually  spoken  in  the 
school  districts  of  the  United  States  (p.21). 

Many  of  these  LEP  students  suffer  repeated  failure  in  the  classroom,  fall  behind  in  grade,  and  drop  out  of  school  if 
they  are  not  provided  services  to  overcome  language  barriers.  Students  who  are  not  proficient  in  English  sometimes 
inappropriately  placed  in  special  education  classes.  Also,  because  of  their  lack  of  English  proficiency,  qualified 
students  often  do  not  have  access  to  high  track  courses  of  Gifted  and  Talented  programs  (U.S.  Department  of 
Education  Office  for  Civil  Rights). 

It  is  suggested  that  concept  maps  be  used  in  vocabulary  learning  to  help  both  ESL  (English  as  Second 
Language)  teachers  and  LEP  students.  Concept  maps  are  a good  way  to  help  teachers  organize  knowledge  for 
instruction  and  a good  way  for  students  to  find  the  key  concepts  and  principles  in  lectures,  readings,  or  other 
instructional  material  (Novak,  1998).  Concept  Maps  are  one  of  the  visual  organizers  being  explored  as  instructional 
and  evaluation  learning  tools.  This  paper  focuses  on  how  apply  concept  maps  as  a tool  in  vocabulary  learning  on  the 
Web. 

What  is  a concept  map? 

Concept  mapping  or  concept  maps,  in  general  form,  have  a long  history  of  being  used  to  support  teachers 
and  learners  to  represent  knowledge  in  graphics.  In  education,  Novak  (1977)  designed  a system  of  concept  maps 
that  has  been  widely  applied  in  the  evaluation  of  students’  learning  in  the  school  system  (Novak  & Go  win,  1984). 
Novak’s  work  was  based  on  the  constructivist  theory  of  learning,  which  holds  that  the  learner  constructs  his/her  own 
knowledge.  The  term  of  a concept  map  often  is  used  as  a form  of  a Web  diagram  for  representing  a knowledge 
structure.  In  other  words,  concept  maps  are  often  used  as  a tool  to  help  teachers  and  students  to  represent  their 
knowledge  graphically.  A concept  map,  in  general  form,  consists  of  nodes  which  represent  concepts  and  links 
which  represent  the  relationship  between  these  concepts  (See  Figure  1). 

Much  research  has  found  that  concept  maps  are  effective  tools  to  evaluate  students’  learning.  Novak  (1991, 
1998)  and  researchers  mention  some  benefits  of  concept  maps: 

• Concept  maps  help  teachers  to  organize  and  present  knowledge  for  instruction  and  students  to  grasp  the  key 
concepts  and  principles  in  lectures,  readings,  or  other  instructional  materials. 
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• Concept  maps  help  students  to  be  better  at  meaningful  learning  and  to  reduce  the  need  of  rote  learning. 

• Learners  needed  to  construct  their  own  maps  and  learn  this  method  of  organizing  their  own  knowledge. 

• Concept  maps  can  be  helpful  to  move  the  learner  from  mere  representational  meaning  to  richer  conceptual 
meaning. 

• Concept  maps  play  a key  role  as  a tool  to  represent  knowledge  held  by  a learner,  and  also  the  structure  of 
knowledge  in  any  subject  matter  domain. 

There  are  a variety  of  maps,  such  as  spider  concept  map,  hierarchy  concept  map,  flowchart  concept  map,  and 
systems  concept  map.  In  this  paper,  hierarchy  concept  maps  have  been  chosen  to  help  LEP  students’  vocabulary 
learning. 

Concept  maps  as  a tool  in  vocabulary  learning 

According  to  Novak  (1998)  concept  maps  are  visual  organizers  that  promote  can  be  used  as  instructional 
and  evaluation  learning  tools  to  promote  meaningful  learning  Through  the  visualization  of  subject  matter  in  a 
format  that  illustrates  the  relationships  between  prior  knowledge  and  new  concepts,  both  the  instruction  and  learner 
are  able  to  organize  and  comprehend  new  concepts  (vocabulary  in  this  case)  in  an  effective  and  efficient  manner. 

The  acquisition  of  new  English  vocabulary,  one  of  the  important  skills  necessary  for  LEP  students, 
frequently  involves  much  representational  learning,  but  the  full  conceptual  meaning  of  technical  vocabulary  may 
take  years,  and  for  some  students  little  more  than  representational  meaning  may  be  achieved.  When  definitions  for 
vocabulary  words  are  learned  by  rote,  representational  learning  does  not  automatically  advance  to  conceptual 
learning.  However,  representational  learning  may  provide  language  labels  that  may  serve  to  facilitate  concept 
learning  (Vygotsky,  1978;  Novak,  1998).  Unfortunately,  much  school  learning  that  should  be  concept  learning  is 
little  more  than  representational  learning  for  many  students.  Students  learn  definitions  for  concepts,  but  they  do  not 
acquire  the  meanings  for  the  concepts.  For  instance,  many  students  learn  that  a “family”  is  any  group  of  people  who 
are  related  to  each  other,  especially  a “mother,”  “father,”  and  their  “children,”  but  learners  often  cannot  explain  what 
that  definition  means  in  ways  that  make  sense  to  them. 

In  this  paper,  the  creation  of  a concept  map  to  help  LEP  students’  vocabulary  learning  (as  shown  in  Figure 
1)  is  suggested. 


Instructional  design  model 


The  purpose  of  the  paper  is  to  introduce  a Web-based  self-instruction  program  that  has  been  designed  to  teach 
English  vocabulary  using  a concept  map  to  LEP  students  in  a way  that  helps  them  fully  understand  the  vocabulary. 
Using  a concept  map,  this  self-instruction  program  is  aimed  at  helping  LEP  students  understand  and  learn 
vocabulary  in  the  context  the  relationship  between  one  word  and  another. 

The  use  of  visual  text  and  graphics  are  some  of  the  most  popular  tools  in  the  Web-based  learning.  Graphics 
are  able  to  represent  important  information  and  are  often  used  for  supporting  text  (Newby,  Stepich,  Lehman,  & 
Russell,  1996).  The  most  widely  used  asynchronous  distance  learning  tool  is  that  of  online  courses  primarily  posted 
in  visual  text  and  static  graphics  (Liles,  2004). 

Hopkins  and  Bean  (1999)  suggested  the  verbal- visual  word  association  strategy  to  help  students  move 
beyond  the  rote  memorization  of  words  and  their  definitions.  In  this  study,  one  self-instructional  module  that 
teaches  what  the  definition  of  “family”  is  for  students  with  multimedia  methodology  is  provided. 

According  to  cognitive  load  theory  (Sweller,  1999)  and  Mayer’s  idea  of  multimedia  learning  (Mayer, 
2001),  the  “modality  effect,”  “replacing  visual  text  with  spoken  text,”  and  the  “cueing  effect,”  “adding  visual  cues 
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relating  elements  of  a picture  to  the  text,”  combine  to  enhance  the  effectiveness  of  multimedia  instruction  (Tabbers, 
Martens,  & Merrienboer,  2004).  In  addition,  some  researchers  have  stated  multimedia  explanation  that  have  “fewer 
rather  than  many  extraneous  pictures  and  words”  is  easier  for  students  to  understand  (Mayer  & Moreno,  2002). 
Mayer  and  his  colleagues  have  conducted  a decade’s  worth  of  research  investigating  the  nature  and  effects  of 
multimedia  presentations  on  human  learning. 

The  Web-based  self-instruction  program  consists  of  combinations  of  text,  audio,  graphics,  animation,  and 
video  on  the  Web.  This  multimedia  presentation  gets  and  holds  students’  interest;  many  educators  believe  this  is 
essential  when  working  with  today’s  video  generation  (Jonassen,  2000). 

The  instruction  in  this  program  is  made  up  of  a lesson  section  (See  Figure  2),  practice  session  (See  Figure 
3,  and  quiz  section  (See  Figure  4)  by  Macromedia  Flash  MX. 


Figure  2.  Illustration  of  frames  of  the  lesson  section 


5 

\ 9 

Who  Am  I? 

"I  am  the  woman  that 
a man  is 
married  to." 


Select  one  best  answer 
Wife 
Husband 
Mother 
Grandmother 

Next 

0 

Figure  3.  Illustration  of  frames  of  the  practice  section 
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Name: 

Direction:  Select  best  answer.  After  finishing  quiz,  please  click  Submit  button. 
You  will  see  your  result. 

1.  Who  Ami? 

"I  am  a very  young  child  who  has  not  yet  learned  to  talk." 

O Baby 
O Father 
O Mother 
O Uncle 

2.  Who  Ami? 

"I  am  a female  parent." 

O Baby 
O Mother 
O Father 
O Grandmother 


Figure  4.  Illustration  of  frames  of  the  quiz  section 

The  lesson  section  consists  of  three  frames  as  shown  in  Figure  5.  First,  the  left  frame  shows  how  to  build  a concept 
map  with  words. 

Presenting  vocabulary  using  a Demonstrating  how  to  pronounce 
concept  Map  words  with  video  clips 


Illustrating  what  the  words  mean 
with  visual  text 


Figure  5.  The  frames  of  the  Web-based  self-instruction  program 

Students  will  be  able  to  see  how  to  connect  one  node,  which  represents  a word,  to  another  node,  which 
represents  one  other  word,  with  a link.  These  visual  maps  will  attract  the  learners’  attention  and  make  them  focus  on 
what  they  need  to  perceive  and  how  they  can  organize  and  link  their  prior  knowledge  (old  vocabulary)  and  the  new 
knowledge  (new  vocabulary).  Next,  the  right-top  frame  shows,  via  video  clip,  how  to  pronounce  the  words.  Students 
will  be  able  to  watch  and  listen  to  this  video.  The  video  will  demonstrate  the  correct  movement  of  the  mouth  and 
lips  in  the  pronunciation  of  the  word.  Finally,  the  bottom-right  frame  shows  what  the  word  means  with  text.  Even 
though  the  definitions  of  words  are  given,  the  emphasis  of  this  approach  is  not  on  rote  learning. 

The  practice  session  is  similar  to  the  lesson  session.  It  will  help  students  to  practice  using  their  knowledge 
through  the  concept  maps.  The  questions  require  students  to  recall  their  knowledge  and  to  apply  new  information  in 
mastery  learning. 

The  final  section,  which  contains  quizzes,  will  help  teachers  to  evaluate  students’  learning  and  prepare  the 
next  lesson  for  learners. 

An  example  of  the  entire  program  can  be  found  at 
http://teacherportfolio.indstate.edu/daesang/ConceptMap.html. 
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Conclusion 

This  Web-based  self-instruction  program  will  aid  LEP  students  acquire  English  vocabulary  in  a meaningful 
and  relevant  manner.  Through  the  use  of  this  program,  which  utilizes  concept  maps  and  includes  three  sessions,  the 
instructor  is  able  to  organize  vocabulary  in  a meaningful  way,  and  learners  can  easily  see  the  relationships  between 
words,  connect  prior  knowledge  to  new  vocabulary,  and  grasp  a deeper  understanding  of  the  vocabulary  in  a 
conceptual  context.  This  is  turn  can  lead  students  to  retain  the  vocabulary  in  a meaningful  manner  and  not  simply 
“memorize”  words  out  of  context.  It  is  also  hoped  that  the  multimedia,  self-instructional  format  of  this  program  will 
attract  the  attention  of  students  and  motivate  them  to  acquire  the  targeted  vocabulary. 
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Abstract 

In  this  paper,  we  examine  the  findings  of  working  memory  toward  the  design  ofweh-hased  learning 
environments.  The  rapid  growth  ofweh-hased  learning  environments  has  raised  various  questions  about  the  design 
of  learner-centered  environments.  The  key  to  developing  effective  learner-centered  environments  is  to  have  a clear 
understanding  of  how  learners'  minds  work.  In  particular,  information  processing  theory  can  help  designers 
support  learners  when  they  process  information  from  instructional  materials.  At  the  center  of  information 
processing  theory  are  memory  models,  especially,  working  memory.  In  this  paper  we  will  pursue  a learner -centered 
approach  hy  using  guidelines  based  on  the  instructional  approach  of  working  memory  theory. 

Introduction 

The  rapid  growth  of  web-based  learning  environments  has  raised  various  questions  about  the  design  of 
learner-centered  environments.  The  key  to  developing  effective  learner-centered  environments  is  to  have  a clear 
understanding  of  how  learners’  minds  work.  Though  constructivism  has  in  recent  years  been  examined  for  its 
implications  for  the  design  of  online  learning  environments,  earlier  theories  still  have  much  to  offer  to  designers.  In 
particular,  information  processing  theory  can  help  designers  support  learners  when  they  perceive,  store,  and  retrieve 
information  from  instructional  materials  (Gillani,  2003).  The  areas  of  information  processing  in  cognitive  theory  are 
important  to  multimedia  design  (Alessi  & Trollip,  2001;  Moreno  & Mayer,  1999). 

Information  processing  theory  has,  through  the  years,  offered  many  insights  into  the  design  of  web-based 
instruction.  At  the  center  of  information  processing  theory  are  memory  models,  particularly  working  and  long-term 
memory.  The  learners  encode  and  retrieve  information  and  they  are  confronted  with  various  problems,  such  as  the 
complexity  of  the  information  structure,  cognitive  overload  in  terms  of  infinite  information,  and  designs  that  fail  to 
consider  individual  differences.  Instructional  design  guidelines  based  on  the  information  processing  theory  should 
inform  the  design  of  web-based  instruction.  In  web-based  learning  environments,  learners  are  required  to  engage  in 
challenging  cognitive  activities,  including  controlling,  shifting,  updating,  retrieving,  and  inhibiting  what  they  see  on 
the  screen.  These  functions  mainly  occur  in  a working  memory,  where  the  major  cognitive  activities  take  place 
(Miyake  et  al.,  2000).  Working  memory  mechanisms  have  fundamental  features  responsible  for  cognitive  processing 
in  general  (Richardson,  1996)  and  are  a main  issue  of  instructional  design  guidelines  in  information  processing. 

Working  Memory  Model 

From  the  perspective  of  information-processing,  learning  involves  the  processes  for  encoding,  storage,  and 
retrieval  activities  of  information  within  the  human  cognitive  system  (Flavell,  Miller,  & Miller,  2002).  Over  the  past 
few  decades,  researchers  have  examined  the  psychological  research  on  memory  processes  based  on  a widely  used 
information-processing  model  (Flavell  et  al.,  2002).  A human  information-processing  model  represented  as  sensory 
register,  short-term  memory,  and  long-term  memory  was  originally  proposed  by  Atkinson  and  Shiffrin  (Atkinson  & 
Shiffrin,  1968).  Also  they  postulated  that  the  short-term  store  could  be  regarded  as  a “working  memory”  which 
transfers  information  from  the  sensory  register  and  the  long-term  store  (p.  90,  92).  The  information  processing 
begins  when  the  sensory  register  receives  the  stimulus  from  the  environment;  then  the  sensory  register  transfers 
information  to  working  memory  (Schunk,  2003),  a system  of  limited  capacity  (Emilien  et  al.,  2004;  Flavell  et  al., 

2002) .  While  information  is  staying  in  working  memory  for  periods  of  10  to  20  seconds,  related  knowledge  in  long- 
term memory  is  activated  and  integrated  with  the  new  information  staying  in  working  memory  (Schunk,  2003). 

Schunk  (2003)  suggests  that  short-term  memory  is  working  memory  and  corresponds  almost  to  awareness. 
Alternately,  even  though  two  terms  of  short-term  memory  and  working  memory  are  typically  used  together,  the 
recent  researchers  postulate  that  differences  exist  between  them  (Emilien  et  al.,  2004;  Kail  & Hall,  2001;  Schunk, 

2003) .  Short-term  memory  and  working  memory  are  both  central  constructs  in  modem  theories  of  memory  and 
cognition.  Short-term  memory  refers  to  information  in  long-term  memory  that  is  activated  above  some  threshold 
(Cowan,  1988,  1995).  Short-term  memory  tasks  generally  require  the  preservation  or  sequential  order  information, 
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whereas  working  memory  includes  short-term  memory  as  well  as  the  attentional  processes  used  to  keep  some  short- 
term memory  contents  in  an  activated  state  (Emilien  et  al.,  2004).  Thus,  it  might  be  suggested  that  short-term 
memory  can  be  thought  of  as  subcomponent  of  working  memory  (Kail  & Hall,  2001).  Working  memory  is 
responsible  for  active  information  processing  rather  than  strictly  passive  short-term  maintenance  of  information 
(Bruning,  2004). 

There  are  several  structural  features  to  represent  the  working  memory.  Baddeley  (1986)  defined  “the 
general  concept  of  working  memory”  in  that  “the  temporary  storage  of  information  that  is  being  processed  in  any  of 
a rage  of  cognitive  task”  (p.  34).  Also,  working  memory  involves  “the  temporary  storage  and  manipulation  of 
information  that  is  assumed  to  be  necessary  for  a wide  range  of  complex  cognitive  activities”  and  plays  a more 
important  role  than  short-term  memory  in  higher  order  cognitive  processes  (Kail  & Hall,  2001).  A number  of 
features  of  models  appear  in  the  contemporary  literature  on  working  memory:  a dynamic  system  (Baddeley  & Hitch, 
1974)  and  a unitary  system  (Engle,  Tuholsky,  Laughlin,  & Conway,  1999)  or  as  multiple  subsystems  each  with  a 
different  capacity  and  function  (Baddeley  & Hitch,  1974).  Unlike  the  traditional  working  memory  described  as  a 
gateway  between  sensory  input  and  long-term  memory  (Baddeley,  1996),  another  approach  proposes  that  working 
memory  operates  as  a working  space  (Emilien  et  al.,  2004) 

Several  significant  working  memory  models  such  as  Baddeley  and  Hitch’s  model  are  still  elaborated  by 
psychological,  neuropsychological  and  neuro-imaging  evidence.  In  the  early  years  of  the  model  by  Baddeley  and 
Hitch  (1974),  the  working  memory  model  consists  of  three  components:  an  articulatory  loop,  holding  speech-like 
representations,  a visuo-spatial  scratch  pad,  holding  imaginal  representations  and  a central  executive,  which  acts  as 
an  overseer  directing  attention  and  coordinating  the  activities  of  the  other  components.  Figure  1 presents  a schematic 
diagram  including  Baddeley’ s working  memory  model.  In  the  current  modified  model  of  working  memory,  the 
fourth  component,  the  episodic  buffer,  is  proposed.  The  episodic  memory  is  a limited  capacity  system  depending  on 
executive  processing  attempts  to  provide  a temporary  interface  between  the  three  components  and  long-term 
memory,  and  integrates  information  form  a variety  of  sources  of  the  subsidiary  system  into  chunks  or  episodes 
(Baddeley,  2000). 


Figure  1 . A cognitive  model  of  web-based  learning 

The  current  research  of  working  memory  is  focused  on  the  significant  theoretical  perspective  on  executive 
functions,  called  the  central  executive,  postulated  by  Baddeley  and  Hitch  (1974).  With  regard  to  the  functions  of  the 
central  executive,  research  on  working  memory  has  been  carried  out.  Executive  functioning  has  to  be  considered  as 
people’s  executive  function  plays  a role  of  allocating  memories  and  conditions  in  a memory  system.  The  executive 
function  in  working  memory  is  thought  to  control  and  regulate  the  flow  of  information  in  a working  memory  system 
(Baddley,  2000).  The  central  executive,  the  core  of  the  working  memory  model,  is  assumed  to  be  an  attentional 
control  system  responsible  for  strategy  selection  and  for  control  and  coordination  of  the  various  processes  in  short- 
term storage  and  more  general  processing  tasks  (Emilien  et  al.,  2004).  That  means  the  central  executive  plays  a role 
as  an  overseer  directing  attention  and  components:  visuo-spatial  scratch  pad,  articulatory  loop,  episodic  buffer. 

Working  Memory  Constraints 

The  cognitive  load  theory  based  on  the  limited  capacities  of  working  memory,  is  being  used  to  facilitate 
learning  using  various  multimedia.  However,  this  paper’s  focus  is  more  on  the  results  of  cognitive  research  related 
to  the  working  memory  model  and  the  sub-components  toward  the  design  of  web-based  learning  environments. 
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All  theories  of  working  memory  assume  its  limitation  in  the  extent  to  which  working  memory  is  able  to 
hold  and  process  information  (Logan,  2004;  Stoltzfus,  Hasher,  & Zacks,  1996).  The  number,  of  items  that  can  be 
recalled  perfectly  in  order,  is  three  to  seven,  depending  on  the  nature  of  items  (Logan,  2004).  According  to  the 
present  theoretical  framework,  the  working  memory  capacity  limit  is  actually  a limit  in  the  capacity  of  the  focus  of 
attention  at  any  one  moment  (Cowan,  2002).  Working  memory  is  supported  by  a limited-capacity  processing 
resource  that  can  be  allocated  to  consistently  meet  the  processing  and  storage  demands  of  cognitive  activities 
(Gathercole,  1999).  Thus,  when  a task  is  encountered  which  exceeds  processing  and  storage  demands,  there  will  be  a 
trade-off  between  processing  and  storage  activities.  Even  though  there  are  many  experiments  to  find  out  evidence 
for  the  trade-off,  evidence  did  not  strongly  support  the  trade-offs  between  processing  and  storage  (Logan,  2004). 
With  respect  to  working  memory,  there  are  various  investigations  in  aspects  of  working  memory  including  storage 
capacity,  amount  of  available  activation,  processing  capacity,  and  ability  to  allocate  resources  to  both  processing  and 
storage  components  (Gathercole,  1999;  Stoltzfus  et  al.,  1996). 

With  regard  to  the  constraints  of  the  working  memory,  people  have  limited  capacity.  Furthermore,  the  short 
duration  of  piece  of  information  within  this  state  is  typically  less  than  one  second  (Farquhar  & Surry,  1995)  if 
people  do  not  pay  attention  to  it.  Even  if  there  are  constraints  in  the  working  memory,  working  memory  is 
considered  as  working  synergistically  with  long-term  memory,  playing  a primary  role  in  control  and  regulation 
function  (Cowan,  1999). 

The  Cognitive  Design  of  Web -based  Learning  Environment 

The  findings  of  information  processing  research  inform  us  about  the  possibilities  imposed  by  the  properties 
of  the  learner’ s information  processing  system.  In  particular,  it  is  focused  upon  the  features  of  each  component  in 
the  working  memory  and  the  improvement  of  the  instructional  design  for  learner-centered  learning  in  web-based 
environments.  Web-based  learning  environments  should  actively  employ  information  processing  theories  because 
they  can  describe  and  prescribe  student  performance  during  learning.  These  instructional  guidelines  based  on 
working  memory  model  and  the  research  findings  are  as  the  follows: 


Chunk  and  simplify  the  instructional  message  as  much  as  possible. 

Sometimes,  we  can  encounter  web-based  learning  consisting  of  complex  structure,  long  context  and 
pictures  in  one  screen.  However  research  has  indicated  that  it  is  necessary  to  make  the  message  simple  and  easy  to 
understand  (Farquhar  & Surry,  1995).  Moreover,  instructions  involving  diagrams  and  text  should  be  restructured 
into  physically  integrated  formats  with  as  small  a number  of  units  as  possible  because  it  needs  to  be  mentally 
integrated  to  be  understood  (Sweller  & Chandler,  1994;  Mayer  & Gallini,  1990;  Kalyuga,  Chandler  & Sweller, 
1999).  Also,  learning  was  enhanced  by  the  elimination  of  textual  material  describing  the  contents  of  a geometric 
diagram  (Tardieu  & Gyselinck,  2002). 

Working  memory  plays  an  important  role  in  the  higher  level  cognition  such  as  comprehending  a text, 
solving  a problem  and  reasoning  while  processing  new  information  (Tardieu  & Gyselinck,  2002).  Especially  in 
problem  solving,  chunking  (or  decomposing)  information,  or  processing  it  is  one  of  the  effective  cognitive 
strategies.  Web-based  learning  is  often  developed  to  improve  student’s  problem  solving  skills  in  specific  domain. 
Learners  in  problem  solving  will  encounter  multiple  kinds  of  information  to  find  a solution.  To  solve  problem,  the 
learner  will  use  various  cognitive  strategies,  such  as  organizing  or  comparing  the  information.  Also,  in  the  complex 
problem  solving,  learner  needs  to  decompose  the  problem  and  analyze  the  sub-problems  and  constraints.  These 
strategies  to  solve  sub  problems  occur  in  the  working  memory  (Sweller  et  al.,  1998). 

Present  instructional  materials  on  the  web  through  multiple  representations 

Many  web-based  instruction  design  still  seems  to  be  determined  by  the  designer’ s aesthetic  consideration 
rather  than  by  first  using  cognitive  learning  theories  to  represent  multimedia  objects.  Even  though  the  use  of 
multimedia  can  be  effective  instructional  format  for  developing  web-based  instruction,  it  is  obvious  that,  if  the 
learner’s  discernible  principles  will  be  basis  of  the  instruction  design,  it  will  fail  to  effectively  increase  learner’s 
working  memory  capacity  during  the  instruction. 

There  are  several  significant  results  of  research  related  to  the  relationship  or  effect  of  working  memory 
capacity  and  multiple  representations.  The  materials  of  web-based  learning  are  basically  developed  using  different 
types  of  media,  such  as  video,  sound,  text,  picture,  diagram  and  so  on.  The  materials  need  to  be  multiple 
representations  and  also  to  correspond  between  text  and  pictures  (Mayer  & Moreno,  1999)  because  working 
memory  has  separate  subsystems — such  as  the  visuo-special  sketchpad  and  phonological  loop  (Baddely,  2002) — 
and  can  perform  the  processes  of  text,  pictures,  and  sound  in  parallel  (Tardieu  & Gyselinck,  2002).  Thus,  if  these 
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components  mentally  integrate  the  information,  the  working  memory  processes  the  information  effectively. 
According  to  Mousavi,  Low,  and  Sweller  (1995),  the  presentation  of  the  instructional  material  using  a mixed  visual- 
auditory  mode  systematically  leads  to  a reduction  in  the  learner’s  acquisition  time,  problem-solving  time,  and 
number  of  errors.  Using  multimedia  materials,  both  the  visual  and  auditory  modalities  can  improve  the  independent 
capacities  of  the  verbal  and  visuo-spatial  working  memory  components,  effectively  increasing  working  memory 
capacity.  Also,  Kalyuga  et  al.,  (1999)  suggested  that  audio-visual  presentation  is  only  effective  when  each  modality 
presents  differing  information  that  can  be  combined  to  assist  learning,  rather  than  the  audio  text  duplicating  the 
visual  text,  because  the  duplication  of  information  is  superfluous  and  places  and  additional  load  on  working  memory 
using  the  audio  and  visual  format  in  computer-based  multimedia  instruction.  Also,  they  suggested  that  when  dealing 
with  split- source  diagrams  and  text,  text  should  be  presented  in  auditory  rather  than  written  form,  text  should  not  be 
presented  in  both  auditory  and  written  form,  and  in  text  written  form  search  for  diagrammatic  referents  should  be 
reduced  by  using  appropriate  markers  or  guides. 

Develop  structure  that  will  easily  iterate  the  learning  process. 

The  web-based  learning  environment  needs  to  be  structured  to  help  the  iteration  of  the  learning  process.  It 
is  possible  to  support  different  strategies  for  the  iterative  learning  activities.  Repeating  information  to  transmit  it 
from  working  memory  to  long-term  memory  is  a process  as  a known  as  rehearsal,  which  provides  a means  for 
saving  information  from  the  forgetting  processes  of  decay  and  interference  (Ormrod,  1999).  The  capacity  of  working 
memory  has  limits  from  pieces  of  information  before  capacity  is  reached  and  from  the  short  duration  within  one 
second  a piece  of  information  can  remain  without  constant  and  quick  rehearsal.  Gathercole  (1999)  postulated  that  a 
large  component  of  the  age-related  change  in  memory  span  in  closely  liked  to  increases  in  the  rate  of  sub- vocal 
articulation.  Increased  rehearsal  rates  lead  to  better  prevention  of  decay  of  temporary  memory  traces,  and  hence  to 
greater  memory  span.  However,  the  sub- vocal  articulation  is  more  effective  older  children  over  about  seven  years  of 
age  and  adults  (Rosser,  1994;  Gathercole,  1999). 

Attract  students’  attention  in  the  web-based  instruction. 

In  web-based  learning,  learners  meet  with  various  representations,  functioning  by  using  media  and  the 
latest  technology.  During  the  information  processing  in  terms  of  learner,  Farquhar  and  Surry  (1995)  posed  that 
students  encounter  various  mental  events  filling  up  the  working  memory  state,  including  learning  events  and  non- 
learning events.  One  of  the  most  important  roles  of  working  memory  is  transmitting  new  knowledge  or  events  into 
prior  knowledge.  To  effectively  transmit  the  events  during  the  working  memory,  high  cognitive  engagement  needs 
to  be  present  between  the  working  memory  state  and  the  long-term  memory  state.  Commonly,  to  reduce  the 
impositions  of  non-learning  events  that  are  not  directly  associated  with  the  learning  process,  it  is  crucial  to  gain  the 
attention  of  the  learner.  The  attention  can  be  gained  and  held  by  changing  stimuli  and  appealing  to  the  learner’ s 
interests. 

The  role  of  attention  affects  the  entire  process  of  information  processing,  such  as,  to  move  sensory 
information  from  the  sensory  memory  into  working  memory  or  to  move  information  from  working  memory  into 
long-term  memory.  One  of  the  experiments  related  to  the  role  of  attention  in  moving  information  from  working 
memory  to  long-term  memory,  Conway  and  Engle  (1994),  reported  that  the  relationship  between  retrieval  and 
working  memory  is  an  individual  difference  in  working  memory  capacity  that  reflects  differences  in  attentional 
resources.  Limited-capacity  controlled  attention  will  play  a role  in  the  tasks.  This  is  consistent  with  the  findings  of 
Baddeley  (1986)  about  a central  executive  as  being  a general  attentional  system.  Conway  and  Engle  (1994)  also 
postulated  that  the  general  attentional  system  is  important  to  the  inhibition  of  superfluous  information  as  well  as  for 
the  activation  and  maintenance  of  information  relevant  to  the  task. 

Ormrod  (1999)  mentioned  factors  affecting  what  people  pay  attention  to  and  what  they  store  in  working 
memory  as:  size,  where  attention  tends  to  be  drawn  to  large  objects;  intensity,  where  more  intense  stimuli  attract 
attention;  novelty,  where  attention  is  given  to  stimuli  that  are  novel  or  unusual;  incongruity,  where  objects  that  are 
incongruous  are  noticed;  emotion,  where  stimuli  with  strong  emotional  associations  capture  attention;  and  personal 
significance,  where  people  tend  to  pay  attention  to  stimuli  in  terms  of  their  personal  importance. 

Elaborate  on  examples  to  draw  consequences  for  practice. 

In  a Stark,  Mandl,  Gruber,  and  Renkl  (2002)  study,  the  authors  recommended  the  form  of  incomplete 
examples  that  should  be  supported  by  additional  explications.  Generally  learners  profit  from  these  kinds  of  examples 
if  they  elaborate  them  and  induce  schemata  relevant  for  problem  solving.  As  the  authors  discussed,  example 
elaboration  decreases  cognitive  interference  and  overload  as  well  as  causing  a positive  effect  to  the  student’ s 
learning  behavior.  Stark  et  al.  (2002)  reported  that  elaboration  training  focusing  on  aspects  of  cognitive  elaboration 
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enhanced  the  quality  of  example  elaboration  and  thereby  significantly  improved  learning  outcomes.  Also  they 
concluded  that  if  the  instructional  goal  is  to  maximize  learning  success,  then  a stronger  support  of  meta-cognitive 
example  elaboration  should  be  taken  into  consideration. 

Be  consistent  with  components  in  the  learning  environments 

A basic  cognitive  skill  is  to  automatically  use  the  brain’s  limited  resources  effectively  (Bruning,  2004).  If 
learner  can  perform  automated  process  during  cognitive  task,  it  requires  very  little  attentional  capacity  in  working 
memory.  The  automaticity  requires  that  repeated  practice  during  activity.  Therefore,  to  design  the  web-based 
learning  environment,  it  is  necessary  to  be  consistent  with  each  component  and  required  activity. 

Consider  individual  differences  in  working  memory  capacity 

One  interesting  model  related  to  working  memory  is  the  activated  portion  of  long-term  memory  (Anderson, 
1983;  Cowan,  1988).  Conway  and  Engle  (1994)  suggested  that  working  memory  capacity  and  the  amount  of 
activation  available  to  long-term  memory  are  equivalent,  though  there  are  differences  among  individuals  about  the 
level  of  activation.  Moreover,  “capacity  or  activation  was  argued  to  change  very  little  with  changes  in  knowledge 
structure”  (p.  355).  Therefore  individual  knowledge  can  increase  in  the  amount  of  information  that  learner  can  store 
and  recall.  However,  a relatively  small  number  of  links  at  each  level  of  the  hierarchy  is  shown  in  individual 
retrieval.  That  means  individuals  show  retrieval  differences,  even  if  learners  have  the  high  working  memory  span.  In 
other  words,  each  learner  has  a different  amount  of  activation  available  to  long-term  memory.  Conway  and  Engle 
(1994)  concluded  that  the  knowledge  units  represented  in  the  working  memory  (as  the  activated  portion  of  long-term 
memory)  would  be  affected  not  only  by  attention,  the  ability  to  keep  information  active,  but  also  by  the  ability  to 
prevent  irrelevant  and  competing  information  from  interfering  with  working  memory. 

In  the  aspect  of  individual  difference,  it  is  necessary  to  understand  age-related  development  of  working 
memory.  The  understanding  of  target  audience’s  cognitive  capacity  in  a web-based  learning  environment  is  also 
important.  According  to  the  research  (Kirschner,  2002),  aging  may  decrease  working  memory’s  capacity,  executive 
function,  processing  speed,  and  learner’s  ability  to  inhibit  irrelevant  or  distracting  information.  The  decline  of 
working  memory  impedes  the  acquisition  of  complex  skills  in  the  following  ways  (Van  Gerven  et  al.,  2002).  First 
due  to  a reduction  of  the  working-memory  capacity,  less  information  elements  can  be  processed,  second,  the  overall 
processing  speed  decreases,  and  a third  decline  concerns  the  ability  to  inhibit  irrelevant  or  distracting  information. 
The  age-related  decrease  of  working  memory  implies  to  instructional  design  for  web-based  learning  environment 
that  web-based  instruction  should  be  carefully  designed  in  an  effective  way  toward  the  complex  information 
depending  on  the  age  of  target  subjects. 


Conclusion 

Web-based  learning  environments  have  become  a popular  learning  delivery  system  for  many  learners.  As 
mentioned  above,  when  the  learning  environment  is  developed,  the  process  of  design  must  focus  on  the  learner’ s 
cognitive  characteristics.  The  effectiveness  of  learner’s  performance  can  be  enhanced  by  considering  the 
implications  of  cognitive  theory.  The  working  memory  model  is  still  work  in  progress.  Researchers  elaborate  model 
in  terms  of  adding  new  components  and  examining  the  characteristics  of  each  component — in  the  working  memory 
model  and  the  relationship  between  learning  activities  and  working  memory  features.  However,  studies  that  examine 
the  effectiveness  of  working  memory  components  for  designing  web-based  learning  environment  and  guide  the 
design  are  lacking  in  the  educational  technology  literature.  Educators  can  better  develop  effective  instruction  using 
working  memory  research.  Continuing  research  will  focus  on  analyzing  the  results  of  instructional  research  of 
memory  to  improve  our  understanding  of  learning  processes  during  attending  web-based  learning. 
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What  Students  do  when  Chat,  Email  and  Discussion  Forum 
are  Available  at  the  Same  Time? 

Xiaojing  Kou 
Indiana  University 

Introduction  and  Purposes  of  the  Study 

There  is  one  important  component  in  distance  graduate  courses,  especially  social  science  courses:  students 
(and  instructor)  discourses  in  forms  of  individual  postings  or  group  discussion.  There  have  been  quite  some  studies 
focused  on  discourses  in  distance  courses.  Most  of  them  focus  on  pedagogical  or  social  issues  (e.g^Fahy,  2002; 
Garrison,  Anderson  & Archer,  2001;  Hara,  Bonk  & Angeli,  2000;  Henri,  1992;  Zhu,  1996).  Few  studies  explore 
media  usage  in  detail.  In  addition,  the  mode  of  communication  studied  is  often  predetermined  and  of  a single  type. 
Even  researches  on  computer  mediate  communication  often  emphasize  on  ‘holding  the  CMC  mode  constant,  while 
varying  the  purpose  of  communication  within  that  mode’  (Herring  & Nix,  1997).  One  characteristic  of  this  research 
is  to  hold  the  purpose  of  the  communication  constant — a team  project  in  distance  education  courses,  while  varying 
the  mode  of  communication — email,  chat  and  discussion  forum.  This  study  thus  explores  situations  when  multiple 
computer-mediated  communication  modes  are  available  and  when  students  have  freedom  to  choose  modes  as  they 
see  appropriate.  The  author  starts  from  reading  and  interpreting  students  discourse  and  then  looks  for  general 
patterns  of  usage  under  different  mode  of  communication  chosen  by  students.  Through  these  efforts,  the  author  aims 
to  not  only  understand  the  diversity  and  focus  of  students  activities  as  expressed  and  shown  from  discourses,  but 
also  interprets  the  choice  and  usage  of  communication  media. 

Research  Question 

The  author  aims  to  answer  the  following  questions:  When  students  use  which  kind  of  communication  mode 
to  perform  what  kind  of  activities  when  multiple  modes  of  communication  are  available  to  them  at  the  same  time? 

Theoretical  framework 

Author  of  this  study  takes  the  perspective  of  a distance  educator  but  mainly  applies  the  theory  of  speech  act 
analysis  as  the  analytical  tool.  Due  to  this  interdisciplinary  effort,  there  are  multiple  theories  that  contribute  to  this 
study. 

First,  in  terms  of  analytical  tools  and  research  methodologies,  this  study  refers  to  speech  act  theory  mainly 
used  in  Discourse  Analysis.  Early  in  1962,  Austin  introduced  the  concept  of  performative  verbs  and  pointed  out  that 
words  can  DO  things.  In  his  book  How  to  Do  Thing  With  Words,  he  listed  five  types  of  perfomative  verbs: 
verdictives,  exercitives,  commissives,  behabitives,  and  expositives.  Philosopher  of  language,  Searl  (1969)  expands 
and  criticizes  work  of  Austin.  More  recently  there  have  been  taxonomies  such  as  that  of  Francis  and  Hunston 
(1992).  Herring  (1996)  studied  two  distribution  lists  and  the  gender  differences  in  terms  of  speech  acts.  Herring  & 
Nix  (1997)  studied  IRC  chat.  They  adopted  Francis  and  Hunston  (1992)’ s category.  This  study  is  more  influenced 
by  Fraser’s  taxonomy  as  introduced  in  McLaughlin  (1984).  Fraser  emphasized  illocutionary  act — which  is  intention 
of  the  speaker.  He  assigned  performative  verbs  into  eight  categories:  asserting,  evaluating,  reflecting  speaker 
attitude,  stipulating,  exercising  of  authority  and  committing.  Compared  to  other  categories,  Fraser’s  taxonomy  is 
more  suitable  for  analyzing  argumentative  discourse,  whose  style  is  closer  to  that  of  the  academic  discussion  in 
education  setting. 

Also  related  to  discourse  analysis  but  in  the  field  of  education,  studies  that  this  study  refers  to  are 
researches  on  classroom  discourse.  However,  these  studies  are  predominantly  on  student-instructor  interaction 
instead  of  student- student  interaction,  which  is  the  case  in  this  study.  A very  influential  study  in  classroom  discourse 
was  done  by  Sinclair  and  Coulthard  (1975).  The  basic  pattern  of  IRF  (Initiation-Response-Follow-up)  has  been 
adopted  by  many  other  researchers  to  study  classroom  discourse. 

Second,  in  terms  of  the  pedagogical  and  social  issues,  knowledge  building  is  considered  as  critical  to 
learning  and  is  much  related  to  this  study.  In  MacKinnon’s  (2000)  study,  he  applied  taxonomy  of  discourse  to 
students’  online  discussion  as  the  basis  of  grading  student  performance.  He  assigned  higher  scores  to  categories  of 
postings  that  are  considered  showing  higher  order  critical  thinking  skills  such  as  compare,  contrast,  idea  to  example, 
example  to  idea,  clarification,  elaboration  and  lower  scores  to  messages  considered  of  relatively  lower  order 
thinking  skills  such  as  only  acknowledging  or  asking  question.  There  are  also  other  studies  focused  on  a variety  of 
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issues  such  as  cooperative  learning  (Harasim,  Calvert,  & Groeneboer,1997),  social  presence  and  cognitive  presence 
(Garrison,  Anderson  & Archer,  1999),  etc. 

Methodology 

The  Data  and  Sample 

The  data  come  from  archived  student  team  communication  in  a graduate  level  distance  course.  A course 
management  system  is  used.  This  system  provides  discussion  forum,  chat,  and  course  email.  Students  have  freedom 
to  choose  any  or  all  of  the  communication  modes.  All  three  types  of  communication  are  archived.  In  order  to  help 
students  understand  theories,  a series  of  problem-based  team  assignments  were  given.  Students  formed  into  small 
groups  of  3 or  4 to  solve  these  problems  or  fulfill  the  task  together.  Two  of  the  teams  were  chosen  for  the  purpose  of 
this  research.  Both  use  all  three  types  of  communication  modes.  Both  teams  get  grade  A for  the  project.  The 
following  table  shows  the  detailed  information  of  the  two  teams  chosen: 


Blue  Team 

Green  Team 

Email 

#Messages 

19 

34 

#Words 

1539 

3372 

#Speech-act  Unit 

119 

258 

** 

5 

#Messages 

75 

526 

#Words 

852 

9728 

#Speech-act  Unit 

123 

863 

# Times 

1 time  (37  minutes) 

4 times  (1  h 34  min,  1 h 40  min,  1 h 1 1 
min,  1 h 21  min) 

5 

5 

£ 

#Messages 

68 

15 

#Words 

6237 

2749 

#Speech-act  Unit 

408 

149 

Total  words 

8628 

15849 

Task 

Debate  the  validity  of  an 
instructional  theory.  The  team 
together  carries  out  a debate  on 
the  pros  and  cons  of  the  theory 
and  should  finally  synthesize  the 
conclusions  of  the  debate. 

To  think  of  an  instructional  goal  that  has,  in 
the  learner’ s experience,  been  particularly 
difficult  for  learners  to  grasp.  The  group 
together  suggests  how  to  use  principles  of 
meaningful  learning  help  design  effective 
instruction  for  this  situation. 

Tablet:  Team  Information 


Data  Collection 

The  data  were  collected  by  another  researcher  in  from  a distance  education  course.  Students’  informed 
consent  was  given  allowing  another  researcher  to  make  use  of  the  data.  All  data  are  sanitized. 

Data  Analysis 

There  are  three  analytical  devices.  The  first  one  is  a simple  time  stamp  analysis.  The  major  analysis  uses 
speech  act  analysis  and  general  themes  analysis. 

Time  Stamp  Analysis : Time  stamps  are  available  for  each  mode  of  communication  archived.  Phases  of 
discussion  will  be  graphed  to  look  for  possible  patterns. 

Speech  Acts  Analysis  and  General  Themes  Analysis : The  basic  unit  of  analysis  is  called  ‘speech  act  unit’. 
The  criteria  for  deciding  a unit  is  to  see  if  it  performs  a separate  act  intended  by  the  speaker  (illocutionary).  The 
samples  are  divided  into  speech  act  units  and  each  unit  is  assigned  a speech  act.  Based  on  a pilot  study  of  speech 
acts  of  100  units  in  the  sample,  the  author  designs  the  following  coding  scheme  revised  from  Fraser’s  taxonomy. 


Speech  Acts  Coding  Scheme  Definition  and  Examples 

Assert: 

State  what  message 
sender  knows, 
observes  or  believes. 
‘Assert’  actions  are 
done  on  the  sender’s 
initiative.  When  the 
sender  provides 
information, 

Notify  assertion  in  advance:  The  message  sender  notifies  in  advance  that  some 
information  or  ideas  will  be  provided. 

Inform:  Provide  information  usually  based  on  creditable  sources.  It  can  include  whole  or 
part  of  an  article,  abstracts,  references,  URLs  (to  resources),  or  paraphrases,  summaries  of 
articles,  books,  instructor’s  words,  information  about  assignments,  or  other  resources. 
They  are  usually  facts  or  other  people’s  ideas.  It  also  includes  the  sender’s  personal 
information. 

Describe:  Describe  phenomenon,  state,  process,  results,  events  or  other  people.  Describe’ 
is  to  give  a clearer  picture  Usually  these  events  or  phenomenon  are  not  long  lasting. 
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describes. . .upon  the 
request  of  other 
teammates,  they 
belong  to  the  ‘reply’ 
category. 

Report:  Report  past  actions,  usually  those  of  the  message  sender,  but  not  limited  to  it. 

Predict/expect:  The  message  sender  predicts  or  expects  a future  state  or  event. 

Restate  (self):  The  sender  restates  one’s  own  statement  in  order  to  emphasize  or  clarify. 

Claim:  Express  (certain  or  uncertain)  ideas,  believes  or  preferences  about  the  subject 
matter  discussed,  the  task  or  external  resources.  Interpret  external  resources.  Evaluate 
one’s  own  previous  statements  or  work.  It  exists  in  the  form  of  statements  or  drafts  and 
submitted  assignments  (can  be  pasted,  attached,  or  put  in  a personal  www  account). 

Hope:  Hope  something  (team  project  related)  good  will  happen. 

Suggest:  The  sender  makes  a proposal  concerning  either  subject  matter  of  the  discussion  or  the  task  at  hand.  It 
suggests  what  the  whole  team  could  do. 

Express  Emotion: 

Express  personal 
emotions  such  as 
sadness,  happiness, 
enjoy,  anger, 
frustration. . . 

Express-positive:  express  positive  emotions  such  as  happiness,  enjoy.  Emoticons  and 
paralinguistic  cues  that  express  happiness  can  be  included. 

Express-negative:  Express  negative  emotion  such  as  anger,  frustration... Emoticons  and 
paralinguistic  cues  that  express  these  feelings  can  be  included. 

Express-neutral:  State  with  a strong  feeling;  Express  surprise;  Exclaim. 

Commit:  Commit  to  an  action,  usually  a task-oriented  action  in  a team  project. 

Restate  (Other):  repeat  or  paraphrase  other  teammate’s  previous  statements. 

Acknowledge:  to  acknowledge  receipt  of  other  teammate’s  message. 

Socialize:  Open/close  a conversation,  pay/maintain  positive  and  negative  face,  lose  one’s  own  face  for  group 
benefit,  be  modest,  show  concern/  sympathy/  understanding/  humor. . . 

Evaluate/ 

Express  Attitude: 

Agree/disagree  or 
qualify/assess  a 
previous  statement, 
done  on  the  sender’s 
initiative. 

Evaluate-positive/agree:  Positively  evaluate  a previous  statement;  accept  a previous 
statement. 

Evaluate-negative/disagree:  Negatively  evaluate  a previous  statement;  disagree  with  a 
previous  statement. 

Other  Remark:  Not  clearly  show  agreement/disagreement.  Look  at  the  discussed  idea 
from  a new  perspective;  add  additional  ideas. 

Inquire:  Requesting  originates  from  speaker  but  elicits  information  or  ideas  or  just  yes/no  answer  from  the 
message  receiver. 

Reply:  the  sender  responds  to  the  ‘inquire’  act. 

Request  Action:  Ask  others  to  do  something.  Can  be  direct  or  very  polite.  Can  be  a difficult  or  an  easy  task.  It  is 
addressed  to  an  individual. 

Respond  to  request  action 

Accept:  Accept  the  request  for  action. 

Decline:  refuse  the  request  for  action. 

Table  2:  Coding  Scheme  for  Speech  Acts 

In  addition  to  speech  act  analysis,  the  author  further  makes  general  themes  analysis.  Different  from  speech 
act  analysis  which  looks  at  individual  activities,  general  themes  are  designed  to  show  the  general  ‘content’  and 
‘theme’  of  conversation.  The  following  is  the  coding  scheme  for  the  general  themes. 


General  Themes  Coding  Scheme  Definition  and  Examples 

Social:  Speech  acts  that  aim  to  maintain  a good  relationship  or  a team  spirit.  Include  socialize,  a very  general 
evaluate  positive  that  does  not  address  a particular  point  in  the  discussion,  and  any  other  speech  acts  that  talk 
about  personal  life  instead  of  the  discussion  focus. 

Central  content  of 
Conversation:  Talk 
about  the  subject 
matter  of  the 
project/  assignment 

Clarify  Assignment  Issues/Discussion  Focus:  Talk  about  understandings  of  the 
expectation  of  the  instructor:  what  is  the  assignment  about,  what  should  be  the  focus 

Theory  Building:  Paraphrase,  summarize  a theory  or  concept;  State  one’s 
belief/understanding  (claim)  of  a theory/concept;  Use  a theory/concept  to  explain 
phenomenon;  Use  examples  to  explain  a theory/concept.  Drafts  or  submitted 
assignments  in  any  forms  are  counted  as  theory  building. 

Content:  Any  other  speech  acts  that  address  the  assignment/discussion  topic  related 
content.  It  can  be  discussion  on  focus  or  structure  of  the  final  delivery. 

Logistics:  Talk  about  team  process,  such  as  when  to  meet,  when  to  submit  the  report. . . (Talks  about  modes 
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of  communication  will  be  counted  separately). 

Technical:  Any  speech  acts  that  address  the  technical  issues  in  the  communication. 

Communication:  Any  speech  acts  that  discuss  choice  or  preference  for  modes  of  communication.  (It  can  be 

part  of  Logistics,  but  are  counted  separately  in  this  category.) 

Table  3:  Coding  Scheme  for  General  Themes 


Results 


Time  Stamp  Analysis  Result 

Based  on  the  time  stamps,  the  two  teams’  posting  present  the  following  pattern. 


A 

Phase  1 

Phase  2 

Phase  3 

Chat 

37m: 

i 

Forum 

1 2 

13  4 27  7 13 

1 

Email 

12  4 2 

1 3 

6 

p 

12  3 4 

5 6 7 8 9 

10  Day 

Figure  l:Green  Team  Result 


i 

Chat 

94mi  lOOmi 

77mi  81mi 

Forum 

1 5 3 

2 4 

Email 

2 12  2 6 

4 

9 8 

messages 

1 2 3 4 5 6 7 8 9 10  11  12  13  14  Day 

Figure  2:  Blue  Team  Result 


Speech  Act  Analysis  and  General  Theme  Analysis  Result 

Results  for  the  Whole  Process 

Frequency  of  each  speech  act  as  and  frequency  of  each  general  theme  are  counted  for  each  of  the  three 
modes  for  each  of  the  two  teams  separately.  The  following  is  a summary  showing  for  each  team  and  each  mode  the 
top  three  most  frequent  speech  acts,  and  the  top  three  most  frequent  general  themes. 


Chat  (%) 

Email  (%) 

Forum  (%) 

Blue  Team 

1.  socialize 

17 

1.  report 

16.8 

1.  claim 

35.8 

2.  inquire 

11.4 

2.  socialize 

14.4 

2.  inform 

15.9 

3.  commit/report  9.8 

3.  claim 

12.6 

3.  report 

10.1 

Green  Team 

1.  inquire 

18.9 

1.  claim 

19.8 

1.  inform 

38.9 

2.  claim 

17.6 

2.  report 

16.7 

2.  claim 

22.2 

3.  socialize 

8 

3.  socialize 

15.9 

3.  report 

10.1 

Table  4:  Top  3 speech  acts 


Chat  (%) 

Email  (%) 

Forum  (%) 

Blue  Team 

1.  social  47.2 

2.  logistics  29.3 

1.  logistics  42.8 

2.  social  24.4 

1.  theory  building  66.2 

2.  content  15.9 
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3.  content  17.9 

3.  content  21 

3.  logistics  10.5 

Green  Team 

1.  content  46.5 

2.  social  18.3 

3.  theory  building  17.7 

1.  content  39.9 

2.  logistics  26.3 

3.  social  23.2 

1 . theory  building  58.4 

2.  content  30.9 

3.  logistics  6.7 

Table  5:  Top  3 general  themes 


Cross  Phases  Findings 

Frequencies  of  each  speech  act  and  general  theme  for  each  mode  and  teams  are  further  counted  in  terms  of 
the  three  phases  as  shown  in  Figure  1 and  Figure  2.  These  results  are  summarized  into  the  following  tables: 


Phase  1 

Phase  2 (%) 

Phase  3 (%) 

Blue  Team 

1.  socialize  17 

2 . inquire/commit  11.4 

3.  report  9.8 

Green  Team 

1.  inquire  20.4 

2.  claim  13.5 

3.  report  12 

1.  claim  23.7 

2.  inquire  16.9 

3 . socialize/  evaluates-  9 . 1 

Table  6:  Chat-Top  3 Speech  Acts  in  Phases 


Phase  1 (%) 

Phase  2 (%) 

Phase  3 (%) 

Blue  Team 

1.  report  16.9 

2 suggest  /socialize 

13.6 

1.  claim  28.6 

2.  suggest  19 

3.  socialize  14.4 

1.  report  23.1 

2.  socialize  15.5 

3.  commit/inquire  10.3 

Green  Team 

1.  claim  17.1 

2.  socialize  16.3 

3.  report  12.4 

1.  socialize  30.7 

2.  notify/  inform/  report 
15.4 

1.  claim  22.9 

2.  report  20 

3.  socialize  14.3 

Table  7:  Email — Top  3 Speech  Acts  in  Phases 


Phase  1 (%) 

Phase  2 (%) 

Phase  3 (%) 

Blue  Team 

1 . report/  claim/  inquire 
23.8 

1.  claim  44.2 

2.  inform  20.5 

3.  report  7.8 

1 . claim/inquire  15.4 

2.  report  13.5 

Green  Team 

1.  inform  45.7 

2.  claim  26.7 

3.  notify  12.4 

1 . report  25 

2.  inform  22.7 

3.  inquire  18.2 

Table  8:  Discussion  Forum — Speech  Acts  in  Phases 


Phase  1 

Phase  (%) 

Phase  3 (%) 

Blue  Team 

1.  social  47.2 

2.  logistics  29. 

3.  content  17.9 

Green  Team 

1.  content  50.4 

2.  theory  building  22.2 

3.  logistics  12.7 

1.  content  41.4 

2.  social  29.6 

3.  logistics  15.9 

Table  9:  Chat— General  Themes  in  Phases 


Phase  1 (%) 

Phase  2 (%) 

Phase  3 (%) 

Blue  Team 

1.  logistics  54.2 

2.  social  25.4 

3.  content  8.5 

1 . logistics/  content 

33.3 

2.  theory  building  19 

1.  content  33.3 

2.  logistics  30.8 

3.  social  28.2 

Green  Team 

1.  content  34.3 

2.  logistics  29.5 

1.  content  61.5 

2.  social  38.5 

1.  content  42.1 

2.  logistics  26.4 
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3.  social  23.8 

3.  social  21.4 

Table  10:  Email — General  Themes  in  Phases 


Phase  1 (%) 

Phase  2 (%) 

Phase  3 (%) 

Blue  Team 

1.  Theory  building  47.6 

1.  theory  building 

88 

1.  logistics 

32.7 

2.  content 

28.6 

2.  content 

9.2 

2.  content 

31.7 

3.  logistics 

14.3 

3.  logistics 

2.1 

3.  social 

19.2 

Green  Team 

1 . theory  building 

66.7 

1.  theory  building  38.6 

2.  content 

28.6 

2.  content 

36.4 

3.  logistics 

2.9 

3.  logistics 

15.9 

Table  11:  Forum  Themes  Phases 


Conclusion 

Conclusion  1 : Medium  choice  and  usage  vary  over  time.  Time  stamp  analysis  shows  both  teams  presented 
similar  patterns — Email  use  spanned  from  the  first  day  to  the  last  day  of  discussion,  with  intensive  use  on  the  two 
ends  of  the  project  and  with  much  less  frequency  in  the  middle  of  the  project.  Discussion  forum  was  rather  heavily 
used  in  the  middle,  though  there  is  team  difference.  Blue  team  used  a lot  of  discussion  forum  and  used  chat  less. 
Green  team  heavily  used  chat  instead  of  forum.  Three  broad  phases  are  thus  identified.  This  result  adds  another 
dimension  to  the  study:  how  phase  of  discussion  interacts  with  acts,  themes,  and  medium. 

Conclusion  2:  The  general  themes  vary  over  phases.  However,  one  persistent  theme  is  “logistics”. 
Communication  tools  that  aim  to  facilitate  student  collaboration  should  be  designed  to  facilitate  team  logistics.  To 
look  across  the  phases,  it  is  found  that  Phase  I and  Phase  III  have  more  social  and  logistics  themes.  Students 
establish  relationship  in  Phase  I.  They  confirm  and  appreciate  the  relationship  in  the  final  phase.  They  also  establish 
team  norms  and  decide  working  communication  modes  in  Phase  I.  They  wrap  up  in  Phase  III.  Phase  II  is  for  theory 
building  and  content  themes,  though  these  two  themes  persist  in  the  final  stage.  While  this  conclusion  might  be 
obvious,  distance  instructors  should  be  conscious  of  this  result  and  be  conscious  of  which  stage  his/her  students  are 
in  and  provide  support  and  guidance  desired  in  the  specific  stage. 

Conclusion  3:  The  frequency  of  speech  acts  varies  over  modes  of  communication.  The  findings  show  that 
the  top  three  frequent  speech  acts  differ  across  modes,  i.e.,  students  do  different  things  in  different  modes  of 
communication.  Students  use  both  chat  and  email  to  socialize,  while  in  discussion  socialize  was  not  an  important 
speech  act.  Students  also  use  chat  when  they  need  an  immediate  answer  to  a question,  thus  “inquire”  emerged  as  a 
top  speech  act.  Students  use  email  to  “report”  both  personal  past  events  and  things  they  have  done  for  the  team. 
Therefore,  the  group  email  is  a good  source  for  a conscientious  instructor  to  know  what  students  are  doing  and  have 
done.  “Inform”,  “claim”  and  “report”  were  main  activities  in  the  Discussion  Forum.  The  forum  seems  to  be  a place 
for  learners  to  provide  information  and  state  opinions,  usually  in  long  paragraphs. 

Certain  acts  that  do  not  surface  the  top  three  also  are  significant  for  understanding  medium  choice.  Chat 
seems  to  be  a place  for  expressing  emotions.  This  may  be  due  to  the  fact  that  immediate  feedback  can  be  received, 
and  students  may  feel  more  of  a personal  touch  when  ‘listening’  and  ‘being  listened  to’  in  chat.  Going  through  the 
chat  archive  may  help  the  instructor  understand  feeling  of  students  better  than  from  other  modes. 

Conclusion  4:  There  are  variations  brought  by  team  working  style.  In  the  two  samples  we  choose,  the  Blue 
team  seems  a cautious  and  organized  team.  They  use  Discussion  Forum  as  the  major  venue  for  discussion.  They  lay 
out  eight  topics  for  the  project,  and  they  respond  to  those  topics  consistently.  The  Green  Team  is  a passionate  team. 
They  make  full  use  of  chat  and  each  time  they  brainstorm  for  more  than  one  hour.  Their  topics  and  focus  can  change 
with  the  chat  going  on.  Different  team  styles  may  be  good  for  different  tasks.  Instructor  should  consider  team  styles 
and  task  characteristics  and  give  responding  guidance,  support  and  feedback. 

Conclusion  5:  Media  choice  and  usage  is  a complex  process.  Choice  of  media  is  complicated  by  the 
combination  of  phases  of  tasks,  communication  mode  characteristics,  and  the  human  usage.  It  is  not  realistic  to 
conclude  on  an  absolute  optimum  choice/usage  of  modes.  Multiple  variables  should  be  considered. 

Contribution  of  this  Research 

Distance  educators  are  faced  with  two  challenges:  the  challenge  of  designing  instructional  strategies  and 
students  activities,  and  the  challenge  of  choosing  or  creating  optimum  technical  environment.  This  research  starts 
from  the  technological  perspective  (more  specifically,  the  communication  media)  and  relates  it  to  pedagogical 
issues.  Technological  and  pedagogical  are  no  longer  separate  considerations  for  distance  researchers.  For  example, 
for  instructors,  asking  students  to  use  certain  mode  of  communication  is  not  only  a matter  of  choosing  the  most 
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convenient  one  or  choosing  the  one  that  the  instructor  is  familiarized  with  or  prefers,  it  should  include 
considerations  of  which  mode  or  combination  of  modes  of  communication  work  best  for  the  students  and  the 
learning  tasks.  For  system  designers,  consideration  for  designing  a discussion  forum  will  include  additional  issues 
such  as  what  type  of  message  labels  can  we  build  into  the  forum  to  facilitate  students  discussion  and  problem 
solving  in  addition  to  the  single  Subject  line.  This  research  also  examines  multiple  media  at  the  same  time  and 
explores  the  complexity  of  media  choice. 

In  addition,  results  of  frequent  students  speech  acts  can  also  be  the  basis  for  designing  more  sophisticated 
grading  rubrics  for  course  that  depend  heavily  on  online  discussion.  Such  rubrics  have  already  been  used 
(MacKinnon,  2000).  However,  it  needs  more  attention  and  further  refinement  based  on  scientific  results. 
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Transparency  in  Education:  Fostering  Active  Participation  in  the  Educational 

Process 


Andrew  E.  Krumm 
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Introduction 

The  purpose  of  this  paper  is  to  describe  why  current  school  reform  initiatives  that  attempt  to  involve 
stakeholders  through  information  disclosure  are  not  working  and  how  transparency  impacts  the  information 
disclosure  process.  School  reform  initiatives  and  policies,  like  the  No  Child  Left  Behind  Act , often  seek  to  promote 
more  involvement  from  parents  and  other  community  members.  These  reforms  forward  notions  of  school  choice 
and  mandatory  reporting  of  standardized  test  scores  as  ways  to  involve  stakeholders  in  the  educational  process. 
However,  these  measures  can  paradoxically  limit  involvement  by  singling  out  what  is  discussed  about  student 
achievement  and  school  improvement  to  the  content  of  what  is  disclosed.  Parents  and  other  community  members, 
i.e.,  stakeholders,  need  more  than  selective  information  in  order  to  become  active  participants.  Active  participation 
requires  access  to  multiple  sources  of  pertinent  information  and  opportunities  to  provide  feedback. 

Corruption  Control 

Schools  are  often  required  by  their  state  government  to  disclose  information  about  their  financial  condition 
and  how  they  use  tax  revenues.  State  mandates  and  federal  law  have  extended  the  disclosure  of  information  to 
student  proficiency  levels,  dropouts,  and  the  qualifications  of  teachers.  These  disclosure  requirements  come  under 
the  umbrella  of  corruption  control  in  order  to  ensure  the  quality  of  education.  Their  purpose  is  to  protect  the 
community  and  inform  consumers  by  allowing  sunshine  to  reveal  any  wrongdoing  or  incompetence.  Corruption 
control  focuses  on  “clarity  of  roles  and  responsibilities;  public  availability  of  information;  open  budget  preparation, 
execution,  and  reporting;  and  assurances  of  integrity”  (Oliver,  2004,  p.  5).  While  there  exists  certain  mechanisms  in 
schools  like  transportation,  food  service,  accounting,  and  payroll  that  can  be  made  transparent  based  on  these 
guidelines,  most  educational  processes  do  not  neatly  fit  into  a traditional  model  of  corruption  control. 

Promoting  transparency  as  corruption  control  is  often  thought  to  promote  better  consumers  through 
disclosing  information  that  can  allow  for  informed  school  choice.  However,  consumers  are  often  passive 
participants  in  the  educational  process  because  they  are  oftentimes  merely  receivers  of  information.  They  are  often 
fed  one-size-fits-all  information  without  the  supporting  details  as  to  what  generated  the  information.  For  example,  a 
standardized  test  score  is  disclosed  to  stakeholders  without  any  differentiation  based  on  audience  and  with  no  link  to 
instructional  practices  and  other  variables  that  may  have  impacted  the  score. 

Transparency  is  also  discussed  in  many  contexts  as  a means  for  achieving  accountability.  However, 
promoting  accountability  often  has  a limiting  end  that  denies  complexity.  Accountability  requires  simplification, 
such  as  a bottom  line  in  business.  For  example,  in  order  to  hold  teachers  accountable  for  student  performances  on  a 
standardized  test  means  to  deny  the  complexity  inherent  in  the  student-teacher  relationship  and  the  numerous 
variables  found  in  a classroom.  Therefore,  general  information,  such  as  a standardized  test  score  or  a grade  for  a 
class  without  the  supportive  details  as  to  what  generated  the  score  or  grade  has  limited  usefulness  because  it  does  not 
encapsulate  the  range  of  variables  in  a given  context.  On  the  other  hand,  specific  or  detailed  information  disclosed 
without  accompanying  generalizations  can  leave  the  stakeholder  bewildered. 

Active  Participation 

While  transparency  as  corruption  control  requires  oversimplification  and  promotes  passivity  on  the  part  of 
stakeholders,  transparency  as  active  participation  embraces  complexity  allowing  for  an  open  process  built  on 
feedback.  Focusing  solely  on  corruption  control  does  not  allow  for  active  participation  on  the  part  of  stakeholders 
because  the  information  often  flows  in  only  one  direction  and  its  content  is  selective  to  only  a few  school  processes. 
Active  participation,  however,  requires  information  with  varying  levels  of  specificity  about  more  educational 
processes  and  expanded  mechanisms  to  provide  feedback.  Because  stakeholders  have  varying  needs,  a process  of 
moving  from  general  to  specific  information  allows  stakeholders  to  retrieve  the  level  of  information  relevant  to 
them.  In  order  for  this  to  occur  connections  are  required  between  different  pieces  of  information.  For  example, 
when  a school  discloses  its  standards  and  benchmarks,  they  could  be  more  useful  to  stakeholders  if  connected  to 
student  work  samples. 
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Transparency  as  active  participation  is  necessary  in  education  because  the  structure  of  schools  is  not 
conducive  to  one-way  flows  of  information  (Weick,  1982).  Weick  (1982)  describes  schools  as  loosely  coupled 
systems  in  that  they  resemble  collections  of  independent  cells  rather  than  a tightly  integrated  bureaucracy.  In  a 
loosely  coupled  system  active  disclosure  is  often  the  only  means  of  allowing  other  stakeholders  to  gain  access  to 
information  about  what  is  going  on  in  other  independent  cells.  Therefore,  the  connectivity  of  information  and  the 
ability  to  provide  feedback  also  impacts  stakeholders  inside  of  schools  ability  to  actively  participate.  Examples  of 
information  that  all  stakeholders  need — both  inside  and  outside  of  schools — are  curriculum  roadmaps,  standards 
associated  with  curriculum,  exemplars  of  student  work,  criteria  for  assessing  assignments,  lessons  plans,  progress 
reports,  discipline  and  attendance  reports,  and  samples  of  past  and  present  student  work. 

The  acquisition  and  dissemination  of  timely  information  can  be  enhanced  through  technology.  In  schools 
these  systems  are  commonly  referred  to  as  administrative  or  student  information  systems.  Classroom  websites  and 
similar  online  environments  are  also  examples  of  resources  that  enhance  participation  in  the  educational  process. 
These  systems  provide  the  necessary  connectivity  through  the  ability  to  link,  or  hyperlink , general  and  specific 
information.  They  also  allow  for  increased  opportunities  for  feedback  through  email,  discussion  groups,  and  blogs. 
As  with  discussions  on  distance  education,  technology  plays  an  important  role  in  transparency,  yet  it  is  what  one 
does  through  the  technology  that  often  deserves  more  attention. 

When  the  aim  of  transparency  is  active  participation,  feedback  is  a distinguishing  quality.  Feedback  is 
generated  through  two-way  communication.  Active  participation  requires  the  exchange  and  interpretation  of 
information  by  all  parties.  For  example,  students,  parents,  and  teachers  must  jointly  interpret  student  performance  in 
the  classroom.  All  three  parties  must  make  sense  of  shared  information  if  all  three  are  going  to  actively  participate 
in  the  educational  process.  The  school  has  a responsibility  to  create  structures  and  environments  that  encourage  the 
free  and  open  exchange  of  information,  such  as  frequent  conferences  and  e-mail.  If  the  environment  is  not 
conducive  to  feedback,  transparency  as  a means  for  active  participation  is  corrupted.  When  information  flows  in 
one  direction,  compliance  rather  than  participation  becomes  the  norm — no  matter  how  much  information  is 
available. 

Transparency  as  active  participation  cannot  be  reduced  to  rules  and  procedures.  It  is  realized  in  a school  as 
a value  expressed  through  a pattern  of  action.  This  pattern  of  action  is  learned  through  trial  and  error.  Stakeholders 
need  structures  of  interaction  that  bring  them  together  to  exchange  information.  However,  each  school  must  learn 
what  information  and  what  structures  of  interaction  induce  active  participation  on  the  part  of  their  stakeholders. 
These  structures  and  forms  of  information  vary  with  different  communities.  The  pattern  is  discovered  in  the  doing 
not  through  an  abstract  programmatic  approach  with  predetermined  plans. 

If  all  stakeholders  are  granted  access  to  the  information  and  are  allowed  to  interact  with  other  stakeholders, 
they  can  respond  to,  ask  questions  of,  or  clarify  the  educational  process.  For  example,  parents  can  monitor  lesson 
plans  to  see  what  is  being  taught,  how  it  is  being  taught,  and  if  students  are  doing  what  is  expected  of  them  outside 
of  the  classroom.  When  students  and  parents  are  granted  access  to  grade  books,  they  can  view  student  progress, 
understand  grading  criteria,  and  balance  outcomes  with  expectations.  With  all  of  this  information,  opportunities  for 
active  participation  are  increased  because  stakeholders  are  better  equipped  with  the  “what”  and  “how”  of  the 
educational  process. 
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Abstract 

This  study  investigated  instructors’  perceptions  of  their  responsibilities , interactions,  and  resources  in  web- 
based  education  in  Korea  and  the  United  States.  Analyses  of  interview  data  showed  that  Korean  instructors 
perceived  their  major  role  as  that  of  a supporter  of  knowledge  acquisition  and  as  a facilitator  of  interaction  between 
students  and  content;  however,  they  viewed  this  role  as  ideal  but  impractical  The  U.S.  counterparts  perceived  their 
role  as  that  of  a moderator  and  viewed  their  interaction  with  students  as  a critical  component  in  web -based 
education.  This  study  investigated  cultural  differences  related  to  web-based  education  and  contributes  to  the  limited 
body  of  knowledge  on  this  aspect  of  web-based  education  in  both  countries.  Also,  this  study  may  help  designers 
create  web-based  instruction  in  relation  to  different  cultural  contexts.  Although  this  study  is  limited  to  nine 
professors  in  the  U.S.  and  Korea,  the  qualitative  interviews  can  be  expanded  to  a large  community  of  online 
educator  and  learner  to  determine  assessment  of  these  experience,  perceptions,  and  attitudes  about  teaching  and 
designing  web-based  courses  online  through  a web-based  delivery  platform.  Knowledge  of  the  distance  education, 
understanding  the  potential  of  using  a web-based  course  and  various  instructional  platforms  for  teaching  in  cross 
cultural  contexts,  and  being  aware  of  the  responsibility  involved  in  teaching  online  course  can  help  facilitate  a 
meaningful  and  positive  learning  experience  for  educators  and  learners  in  both  countries.  Future  studies  are 
strongly  recommended  to  broaden  the  understanding  of  multi-level  cultural  contexts  and  their  role  in  web-based 
education. 


Introduction 

The  convergence  of  technological  and  pedagogical  developments  has  helped  a new  paradigm  of  teaching 
and  learning  emerge  (Vonderwell,  2003).  Web-based  education,  in  general,  refers  to  delivering  instruction  via  web- 
based  resources  and  communications  to  students  who  are  at  a relative  distant  geographical  location.  As  current 
technology  speeds  up  the  connection  of  physically  separated  learners  and  instructors,  web-based  education  is 
becoming  a main  education  delivery  format  throughout  many  different  countries  including  Korean  and  the  United 
States. 

As  Walsh  and  Reese  have  noted,  “web-based  distance  education  employing  the  new  educational 
technologies  can  improve  the  quality  of  a university’s  educational  offering,  provide  economic  benefits,  and  offer  a 
strategic  advantage  in  piercing  new  markets”  (Walsh  & Reese,  1995).  Many  distance-delivered  university  programs 
that  integrated  with  technologies  can  offer  the  adult  learners  a number  of  opportunities  (Hara  & Kling,  1999). 

However,  the  instructional  intent  and  the  ways  to  apply  web-based  education  would  not  necessarily  be 
identical  among  institutions  in  different  countries  since  educational  results  may  be  influenced  by  cultural 
differences.  Consequently,  the  web-based  learning  environment  would  take  a unique  form  depending  on 
characteristics  of  the  students,  faculty,  and  institutions  in  each  country.  Over  recent  years  there  has  been  a dramatic 
increase  in  the  number  of  Korean  universities  that  use  web-based  resources  in  undergraduate  courses.  This  implies 
the  Web-based  education  has  become  an  important,  independent  component  in  the  Korean  education  system  just  as 
it  has  in  the  U.S.  Given  this  widespread  development,  this  study  examined  how  the  practices  are  implemented  in  the 
two  countries  and  how  the  instructors  from  each  perceive  the  practices  of  their  respective  web-based  education 
systems. 

The  investigators  interviewed  instructors  currently  involved  in  web-based  education  delivery  in  either 
Korea  or  the  U.S.  to  determine  what  their  relative  practices  in  web-based  education  and  also  determine  how  they 
perceive  their  responsibilities,  interaction,  and  resources.  Examining  instructors’  perceptions  provided  an  in-depth 
understanding  of  the  similarities  and  differences  between  the  web-based  education  practices  of  the  two  countries. 

The  purpose  of  this  study  is  to  examine  perceptions  of  instructors  in  Korea  and  the  U.S.  related  to  their 
instructional  practices  while  delivering  web-based  education.  Mainly,  the  comparisons  were  focused  on  aspects  of 
instructor’s  role,  interaction,  and  supporting  instructional  resources. 
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Methodology 


Subjects 

The  participants  of  this  study  consisted  of  nine  instructors  from  Korea  and  the  U.S.  who  had  an  experience 
of  web-based  education  at  least  for  two  semesters.  Five  Korean  instructors,  four  males  and  one  female,  ranging  from 
thirties  to  forties  in  age,  were  interviewed  over  the  telephone.  Four  U.S.  counterparts,  three  males  and  one  female, 
ranging  from  forties  to  sixties  in  age,  were  interviewed  in  person.  All  the  Interviews  were  audio-taped  and 
transcribed.  The  interview  length  was  about  one  hour  thirty  minutes  The  Korean  instructors  were  from  four 
universities  located  in  three  geographically  different  areas  of  Korea  whereas  all  the  U.S.  instructors  were  from  a 
southeastern  University.  All  participants  were  teaching  undergraduate  courses  in  education  related  studies  such  as 
higher  education,  educational  technology,  educational  human  resource  development,  computer  education,  special 
education,  and  English  writing. 

In  order  to  interview  the  instructors,  semi- structured  interview  questions  were  developed.  To  answer  the 
following  questions: 

(1)  How  the  instructors  perceive  their  role  or  responsibility  in  web-based  education?; 

(2)  How  the  web-based  interaction  is  practiced  in  each  country?;  and 

(3)  What  kind  of  resources  the  instructors  use  to  enhance  communication  in  web-based  education? 

Data  Analysis 

The  content  analysis  method  is  used  to  investigate  the  instructors’  perceptions  of  their  responsibilities, 
importance  of  interactions,  and  efficacy  of  resources  in  their  own  web-based  education  system.  According  to  the 
Rubin  and  Rubin  (1995)  approach,  the  textual  data  were  read  by  all  three  researchers  and  discussed  and  then  the  key 
concepts  and  ideas  were  categorized.  The  similar  categories  were  combined  into  broader  ideas  to  be  analyzed. 

Findings 

Instructors’  Role 

From  the  initial  analyses,  it  was  found  that  there  were  several  differences  and  similarities  between 
instructors  from  Korea  and  the  U.S.  in  perceptions  toward  their  role,  interaction,  and  resources  in  web-based 
education.  All  of  the  instructors,  in  general,  considered  their  roles  to  be  a course  developer,  a course  manager,  and  a 
learning  facilitator. 

The  instructors  from  both  countries  thought  their  role,  as  content  developer  was  important.  One  Korean 
instructor,  commented,  “It’s  important  that  an  instructor  develops  online  contents  in  effective  way  and  delivers  well- 
constructed  online  materials  for  target  learners.”  He  continued,  “I  think  it  is  important  for  the  instructors  to  be 
responsible  for  their  role  as  a developer  of  their  instruction  contents  using  authoring  tools  if  we  pursue  the  quality  of 
education,  even  though  it  is  so  much  burden.”  All  five  Korean  instructors  responded  that  they  perceived  that 
developing  their  instructional  materials  is  critical  for  their  instruction  and  for  learners  to  manage  the  high  quality  of 
their  class  delivery  although  they  were  often  limited  by  time  and  resources. 

The  U.S.  instructors  also  perceived  the  importance  of  role  as  a content  developer.  Dr.  B,  a U.S.  instructor 
from  educational  human  resource  department,  expressed  that  his  major  role  was  “to  design  a course  that  is  usable  for 
students.”  Another  U.S.  instructor  also  similarly  noted,  “I  am  responsible  for  the  content,  organizing  it,  writing  it, 
and  preventing  it.  Basically  as  an  instructor  you  have  to  do  all  the  curriculum  design,  all  the  course  design.  And,  you 
also  have  to  make  sure  that  you  have  materials  and  orders.  On  the  web-based,  you  usually  work  with  someone  who 
has  technical  expertise.” 

The  instructors  from  both  countries  reflected  that  one  more  major  role  of  the  web-based  education 
instructor  was  a course  manager  and  a learning  facilitator.  Korean  instructors  noted  that  “. . .the  responsibility  is  not 
only  managing  web-based  course  but  also  engaging  students  in  discussing  and  giving  feedback.”  and  “. . .instructors 
support  learners  for  their  participation  in  their  class  and  encourage  them  to  learn  course  contents  and  information.” 
Dr.  C from  U.S.  stressed  that  one  of  his  major  responsibilities  is  to  facilitate  his  students,  adapting  to  their  individual 
needs  while  their  doing  assignments  or  projects: 

So  my  role  is  in  the  facilitating  part,  face  to  face.  I tried  to  prepare  them  for  the  assignment  and  the  project 

that  they  had  online  and  then  monitored  their  discussions  and  evaluated  it.. . . But  the  way  that  I teach,  I 

believe  this  is  very  important,  you  are  able  to  adapt  and  go  off  into  areas  to  adapt  to  the  students. . . 
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Dr.  D from  U.S.  also  indicated  his  role  as  a facilitator  or  coach.  He  stated,  “I  try  to  design  my  classes  based 
on  learner-centered  instruction  design.  So  I think  one  of  the  major  role,  I think,  it  would  be  more  like  a coach  for 
learning.” 

Interestingly,  the  instructors  showed  different  views  on  what  they  want  to  gain  by  the  web-based  education 
and  how  they  think  the  ideal  role  of  instructor  in  web-based  education.  The  Korean  instructors  perceived  their  role 
more  as  a supporter  for  knowledge  acquisition.  For  instance,  one  Korean  instructor,  Dr.  H,  who  had  lots  of 
experiences  of  contents  and  system  evaluation,  noted  that  many  Korean  instructors  involved  in  web-based  education 
tended  to  evaluate  their  students  in  terms  of  knowledge  acquisition,  saying,  “both  instructors  and  learners  focus  on 
knowledge  and  information  in  their  subject  contents.”  On  the  other  hand,  the  U.S.  counterparts  considered  that  they 
were  primarily  responsible  for  establishing  the  learner-centered  learning  environments  by  mostly  assisting  their 
students’  engaging  in  their  class  activities  and  interactions. 

From  the  responses  of  the  participants  to  the  question  asking  their  role  in  web-based  education,  it  was 
found  that  Korean  instructors  considered  their  role  more  as  a traditional  instructor  that  who  focuses  on  delivering 
knowledge  on-line.  For  example,  Korean  instructor,  Dr.  I,  indicated  that  one-way  videos,  either  synchronous  or 
asynchronous,  had  been  one  of  the  popular  delivery  methods  for  web-based  education  in  Korea.  On  the  other  hand, 
the  U.S.  instructors  viewed  their  role  more  as  a moderator  for  enhancing  student  interaction.  However,  both 
countries  instructors  thought  that  the  good  communication  and  well-organized  courses  are  important  facets  in  web- 
based  leaning  environments  to  make  learners  satisfied.  The  respondents  from  both  groups  also  considered  that  time 
management  was  the  most  challenging  task  when  developing  materials  and  while  monitoring  students  during  course 
related  activities. 

Interaction 

Regarding  interaction  and  integration,  instructors  from  both  countries  shared  that  enhancing  interaction 
was  a key  factor  for  students’  learning.  However,  they  differed  in  the  extent  they  actually  focus  on  how  to  foster 
students’  interaction. 

Interaction  perceived  as  a learning  opportunity 

All  four  U.S.  instructors  thought  that  interaction-integration  to  on-line  instruction  was  an  essential  part  for 
learning.  And,  they  concentrated  more  on  activities  designed  to  promote  students’  interaction.  A faculty  from  the 
special  education  program  in  U.S. , commented  as  the  following: 

. . .1  am  happier  with  interaction  of  my  students.  I can  see  them,  I can  read  them,  and  I can  redirect  them  so 
that  they  do  much  better  jobs.  Some  of  my  students  really  like  on-line  discussion.  And,  some  of  my 
students  participate  in  on-line  discussion  but  they  don't  participate  verbally  in  class.  So,  on-line  discussions 
give  some  of  my  students  an  opportunity.  I can  give  them  more  choices... 

Similarly,  another  U.S.  instructor  indicated  that  interaction  in  on-line  learning  was  crucial: 

. . .1  really  don't  believe  that  you  can  conduct  a really  good  web-based  course  without  interaction. 

Interaction  requires  the  U.S.  professors  to  build  many  factors  and  manage  that  interaction.  A lot  easier  just 
to  bring  assignments  upon  the  Web  and  let  their  students  complete  them  as  they  want  and  grade  the  papers. 
But  there  are  rich,  I think,  educational  experiences  that  are  playing  interaction  and  straight  discussion  on 
particular  topics.  Maybe  creating  things,  like  streaming  media  kind  of  examples  actually  challenge  the 
students  to  go  deeply  into  the  content. . . 

Interaction  in  an  on-line  learning  environment  was  also  perceived  as  enforcement  for  active  and  reflective 
learning.  A faculty  from  U.S.,  Dr.  B,  emphasized  the  effect  of  integrating  interaction  as  the  follows: 

...  An  on-line  environment  can  enforce  students  to  engage  more  actively  to  do  or  reflect  to  do  their  work, 
or  expose  themselves  to  what  they  are  doing,  that  part  of  experience.  In  an  impersonal  learning 
environment,  sometimes  student  don’t  think  that  part  as  a learning  experience.  I think,  that  kind  of  way,  an 
interactive  class  facilitates  web-based  tool  support  of  the  impersonal  class  environment  is  helpful  as  well. . . 

One  U.S.  instructor,  Dr.  D,  regarded  that  integrating  interactions  can  enhance  cognitive  thinking  and 
higher- order  thinking  of  students: 
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. . .It  depends  on  all  variables  including  learning,  students,  its  like,  depends  on  the  concept  you  are  actually 
training  for  at  that  point.  But,  for  me,  I think  what  I would  like  to  get  out  of  interaction  is  better  grasping 
for  the  content  by  the  learners  and  ability  to  apply  that  knowledge  and  higher-order  thinking... 

Korean  instructors  had  similar  agreement  as  their  US  counterparts  considering  interaction  as  a crucial  part 
of  on-line  instruction  and  learning.  One  Korean  faculty  stressed  the  importance  of  interaction  and  motivation  in  on- 
line class  as  the  follows: 

. . .1  realized,  from  my  experiences  of  content  evaluations  on  many  web-based  course  materials,  those  lots  of 
course  materials  were  developed  with  the  interaction  or  motivational  components  ignored.  Thus,  on-line 
delivery  is  often  too  monotonous  and  not  much  motivational.  Therefore,  I think  what  we  should  emphasize 
is  to  support  integrating  interaction  between  students  and  course  materials.  The  next  thing  that  we  should 
focus  on  is  motivating  the  participants  to  maintain  their  interests  for  the  course. . . 

Similarly,  another  Korean  instructor  addressed,  “. . . I think  what  we  should  emphasize  is  to  support 
integrating  interaction  between  students  and  course  materials.  The  next  thing  that  we  should  focus  on  is 
motivating  the  participants  to  maintain  their  interests  for  the  course...”  Also,  one  instructor  from  a Teachers’ 

College  in  Korea,  emphasized  importance  of  integrating  interaction  in  her  course  saying,  “I  think  active  engaging 
in  discussion  is  very  important  for  on-line  learning  and  I really  think  interactive  discussion  impacts  on  learning”. 

Unlike  the  U.S.  counterparts,  the  Korean  instructor,  Dr.  I,  addressed  a concern  in  integrating  interaction 
to  their  class  as  seen  in  the  following  comment: 

. . .But,  in  reality  I saw  many  of  my  students  were  reluctant  to  engaging  in  discussion.  I think  it’s  because 
they  didn’t  feel  comfortable  in  expressing  their  ideas  in  a logical  way,  freely  asking  any  questions  to  me 
or  each  other.  Probably  they  don’t  think  seriously  on-line  discussion. . . 

More  interestingly,  the  Korean  instructors  seemed  to  somewhat  doubt  that  on-line  interaction  can  fully 
follow  the  positive  effect  of  the  face-to-face  interaction.  He  clearly  addressed  as  the  follows: 

. . .1  think  discussion  itself  does  not  increase  learning  achievement.  When  we  focus  on  web-based 
discussion,  understanding  of  concepts  and  principles  may  be  ignored.  Conceptual  understanding  should 
be  supplemented  in  a face-to  face  format.  I would  stick  to  my  position  that  understanding  of  concepts  is 
critical  for  achievement.  Without  the  understanding  of  the  basic  concepts,  the  discussion  is  likely  to  be 
discussion  for  discussion. . . 

It  is  also  revealed  that  most  Koran  instructors  focus  more  on  the  interaction  between  an  instructor  and 
students  than  other  types  of  interactions.  They  employed  face-to-face  interaction  along  with  the  web-based  format 
for  better  communication  between  themselves  and  students.  Dr.  I said  that  as  the  follows: 

. . .what  is  more  important  is  to  discuss  with  students  on  the  questions  and  assignments  which  was  posted  on 
the  web. . ..What  I do  is  to  use  the  web  for  adding  materials.  I think  the  most  important  thing  is  to  meet  with 
my  student  and  providing  instructions  to  them... 

Interaction  applied  in  the  classroom 

The  U.S.  instructors  emphasized  the  importance  of  the  social  aspects  of  course  activities  and  learning  styles 
in  designing  their  instruction,  and  they  invested  lots  of  efforts  to  enhance  learning  through  discussion.  On  the  other 
hand  the  Korean  instructors  indicated  that  a decent  level  of  interaction  is  desirable  for  more  individual  learning  but  it 
is  not  practical  with  the  existing  constraints  such  as  the  large  size  of  classes,  students’  reluctance  to  participate  in 
interactive  activities,  and  limited  support  resources. 

Most  U.S.  instructors  noted  that  they  focus  on  encouraging  students’  discussion.  A U.S.  faculty  stated  as  the 
follows: 

. . .Mostly,  I encourage  my  students  to  interact  with  discussions.  Reflection  component  in  distance 
education  is  perhaps  strength.  In  an  impersonal  learning  environment,  that  writing  and  doing  their 
reflection  are  not  always  comfortable  to  undergraduates. . . 
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Another  U.S.  instructor  noted  about  instructional  strategies  of  using  student’s  role  as  moderators  in  order  to 
enhance  the  interaction  during  their  class  discussions: 

...I  usually  assign  individuals  to  moderate  discussions.  ..They  actually  have  on-going  discussions  by 
teams. . . Students  have  to  moderate,  know  the  discussion,  and  come  ahead  of  time  putting  help,  looking 
questions,  summarizing  discussions... 

In  contrast,  none  of  the  Korean  instructors  mentioned  about  the  employment  of  student  facilitators  for 
discussions.  When  asked  how  participants’  interaction  could  be  enhanced,  all  of  the  U.S.  instructors  from  U.S. 
emphasized  it  was  possible  by  taking  into  account  the  social  aspects  of  activities  and  students’  learning  styles  in 
designing  their  on-line  courses.  He  noted  as  the  follows: 

. . .1  think  there  is  also  a function  of  learning  styles.  We  also  need  to  decide  how  students  do  best  that 
interaction.  I think  that  you  have  to  know  learning  styles  exist  when  you  design  your  course  so  that  you  can 
design  interaction  into  the  course.  I think  it  works  a lot  better  in  some  ways  that  interaction  between  a 
student  and  a student.  We  know  that  education  in  a lot  ways  is  a social  experience.  When  you  get  into  an 
on-line  situation,  we  take  the  social  aspect.  What  we  have  to  do  is  to  design  that  social  backup. . . 

Similarly,  the  U.S.  instructor,  Dr.  B,  indicated  that  how  learning  styles  and  social  aspect  of  participants 
were  included  in  his  course  designing: 

. . .Learning  styles  are  also  important  to  decide  the  best  interaction  in  designing  a course  and  delivering 
the  course  among  students.  What  we  are  trying  to  do  is  students’  interaction  and  we  know  that  as  a social 
experience...  so,  students  more  often  learn  with  engaging  than  others.  They  develop  on-line  relationships 
highly  engaging.  And,  they  have  more  opportunities  to  learn  better. . . 

The  U.S.  instructors  emphasized  the  social  aspects  of  course  activities  and  learning  styles  in  designing 
their  instruction,  consequently  focusing  more  on  student- to- student  interaction,  whereas  the  Korean  counterparts 
appeared  to  focus  more  on  student-to-instructor  interaction.  For  example,  one  instructor  from  the  Seoul  University 
indicated  his  primary  emphasis  was  providing  immediate  feedback  to  students’  performances.  Students-  to- 
content  interaction  and  interactive  discussion  among  participants  were  not  practical  in  his  particular  instruction 
situation  in  which  most  of  participants  are  a full  time  employee  and  the  course  is  delivered  fully  on-line.  He  stated 
as  the  follows: 

...  I think  student- to-content  interaction  is  not  practical  in  Korea.  More  than  70%  of  my  students  have  full 
time  jobs  and  most  of  them  like  to  understand  what  a teacher  instructs  just  like  the  way  they  do  attending  in 
an  off-line  class  even  though  web-based  instruction  allow  us  to  induce  a dynamic  interaction  using  multi 
media.  Some  technical  procedures  that  some  interaction  objects  requires,  for  example,  click  events,  say, 
clicking  buttons  for  turning  to  the  next  frame,  and  text  areas  used  for  asking  personal  opinions  or  ideas,  the 
procedures  of  these  interaction  components  rather  bothers  the  students.  The  student  felt  inconvenient  and 
hard  with  them.  I prefer  to  allow  them  to  access  to  my  instruction  in  one-  way  audio  and  video  format. . . 

Constraints  in  integrating  interaction 

The  instructors  from  U.S.  addressed  a few  challenges  in  managing  students’  interaction  in  their  web-based 
distance  class.  The  biggest  challenge  that  the  instructors  had  faced  was  time  managing  in  dealing  with  lot  of 
workload  resulting  form  monitoring  participants.  For  example,  one  instructor  from  the  Educational  Human  Resource 
program  indicated  as  the  follows: 

. . .Maybe  creating  things  like  streaming  media  kind  of  examples,  you  can  actually  challenge  the  students  to 
go  deeply  into  the  content.  So,  the  biggest  challenge  is  actually  being  able  to  construct  that  kind  of  time  you 
can  give  to  the  students.  Time  management  issue  is  important. . . 

Similarly  another  instructor  from  the  same  program  stated  the  time  managing  problem  as  the  follows: 
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. . .The  most  challenging  part  of  that  is  keeping  tractable  because,  strictly  on-line,  there  is  over- 
communication and  too  much  information  between  students  and  contents  and  questions.  It  gets  to  be  the 
point  where  I usually  setup  a discussion  group,  like  a chat  room. . . 

Lack  of  technical  support  was  also  considered  to  be  a constraint  in  delivering  a web-based  instruction.  A 
U.S.  instructor  indicated  that  one-to-one  technical  guide  for  posting  her  instructional  material  would  be  a lot  of  help 
for  her  web-based  instruction.  He  as  the  follows: 

. . .What  I need  is  that  somebody  walks  down,  says  that  it  is  slow,  or  what's  happening,  and  comes  down 
and  fixes  it.  I need  somebody  proactively  thinking  ahead  of  me.  You  know  I really  don't  know  that  much 
about  technology.  Web-CT,  I want  the  technology  is  transparent,  and  I want  it  to  be  smooth.  I don't  want  to 
think  about  why  it  works  or  doesn't  work.  When  I press  a button  it  works.  That's  all. . . 

All  Korean  instructors,  otherwise,  indicated  participants’  inattention  to  the  active  involvement  in  interactive 
discussions  as  a serious  challenge  in  integrating  interaction  into  their  web-based  courses.  The  large  class  size  was 
also  perceived  as  a constraint.  The  instructor  from  Seoul  Digital  University  reported  that  an  on-line  class  has 
normally  around  one  hundred  students.  The  normal  class  size  for  the  courses,  which  are  offered  for  freshmen  in  his 
university,  mounted  to  as  large  as  400  to  500  of  students.  He  added  that  it  was  very  hard  to  monitor  the  quality  of 
interactions  of  that  many  students. 

Technology  Support  and  Resources 

Three  major  topics  were  identified  in  this  study:  content  development  and  instructor’s  challenge,  the  use  of 
communication  tools,  and  limitations  of  the  technology. 

Content  Development  and  Challenge 

It  is  assumed  that  one  of  the  roles  of  web-based  education  instructors  was  guiding  students  through  the 
contents  developed  by  the  instructors  themselves.  Although  they  were  agreed  with  the  role  as  content  developer  or 
manager,  instructors  from  both  countries  reported  that  they  have  difficulties  to  play  the  role.  Also,  there  are 
emerging  issues  related  to  the  use  of  technology  in  developing  content.  For  example,  an  instructor,  Dr.  D,  noted  that 
“when  we  develop  web-based  materials,  I pretty  much  develop  all  of  them,  but  I also  have  the  GA  help  me.”  A 
Korean  instructor  also  said  “Commonly,  the  instructors  in  charge  of  web-based  courses  develop  the  contents  by 
themselves,  so  an  instructor  and  a content  developer  can  not  be  separated  in  their  role.” 

However,  when  materials  are  developed  to  be  delivered  on  the  Web,  interestingly,  it  seemed  that  the 
Korean  instructors  were  more  aware  of  the  way  of  implementation  and  design  of  the  content  presented  to  students 
on  the  Web.  Otherwise,  the  U.S.  instructors  were  focused  on  content  itself  and  presented  text-based  contents.  The 
sense  one  has  here  is  of  student’s  necessity.  That  is,  the  use  of  new  media  technology  with  contents  on  the  web  is 
perceived  a necessity  by  instructors  of  both  countries.  A Korean  instructor  commented  the  student’s  preferences  on 
the  web-based  learning  activities  were  “most  of  the  students  want  traditional  lecture  form  on  the  web-based  class. . .. 
Students  want  to  listen  to  professor’s  voice  or  video  recorded  lecture,  along  with  contents  on  the  screen,  rather  than 
just  see  text-based  materials.” 

Both  countries’  instructors  indicated  that  the  process  of  delivering  the  materials  is  little  different  between  a 
web-based  and  a face-to-face  form.  An  U.S.  instructor  noted,  “Last  couple  of  years,  I have  modified  material, 
produced  PowerPoint,  produced  pages  of  document,  and  then  I pull  those  together.”  She  continued  “I  did  pretty 
much  doing  the  same  things  as  I do  in  a face  to  face  classroom.”  A Korean  instructor  also  responded,  “I  am  taking 
charge  of  a humanity  class.  The  class  is  focusing  on  transferring  knowledge  to  students.  I think,  therefore,  when  I 
had  transformed  the  class  to  the  web-based  class,  there  are  little  difficulties.”  In  the  process  of  content  development, 
a technical  issue  was  considered  to  be  the  most  difficult  challenge  for  developing  the  material  to  most  instructors  in 
this  study.  For  example,  it  indicated  that  the  lack  of  authoring  tools  to  transform  into  tradition  format  of  material, 
insufficient  technical  ability  and  experience  of  instructors,  and  a time-intensive  process. 

Instructors’  willingness  to  develop  their  own  course  materials  has  been  well  responded  by  the  authoring 
tools  in  the  market  that  has  been  getting  more  user-friendly.  However,  the  instructors  appear  to  have  a hardship  in 
developing  materials  by  the  lack  of  the  clear  directions  with  the  possessed  tools  and  with  the  lack  of  desired  tools, 
which  is  well  reflected  in  the  following  statement  given  by  a U.S.  instructor: 

. . .The  difficulty  is  deciding  within  what  resources  we  have,  what  we  are  going  to  do,  and  what  we  can  do. 
In  going  back  to  your  questions,  before  you  might  decide  in  this  particular  course,  because  we  don’t  have 
the  tools  that  we  need,  we  might  have  to  decide  to  teach  in  a traditional  format. . . 
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With  respect  to  software  to  be  used  in  developing  course  materials,  the  U.S.  instructors  mentioned  that  they 
commonly  use  Adobe  PDF  writer,  Microsoft  PowerPoint,  and  video  editor.  Comparatively  the  Korea  instructors 
indicated  that  they  often  use  GVA  (Global  Virtual  Academy)  of  Youngsan  Information  and  Communication,  Active 
Tutor  of  4Gsoft,  Macromedia  Flash,  and  video  editing  software.  The  Korean  instructors  indicated  that  they  prefer 
using  domestic  authoring  tools  in  order  to  develop  materials  for  the  web-based  education.  Dr.  H commented,  “in  the 
Korean  market,  there  are  so  many  solutions  to  develop  multimedia-based  contents  because  instructors  are  interested 
in  developing  contents  rather  than  the  LMS  (Learning  Management  System)  itself.” 

Communication  Tools 

Embedding  communication  technologies  throughout  the  instruction  system  is  an  important  consideration 
along  with  what  to  include  for  the  content.  When  asked  what  they  were  using  communication  tools  in  the  LMS 
system,  most  instructors  mentioned,  discussion  board  and  email.  Even  though  there  are  more  functions  to  support 
communication  between  instructor  and  students,  they  were  mostly  used  to  general  asynchronous  functions. 

We  could  find  the  differences  between  both  countries  regarding  to  the  uses  and  perceptions  toward  the  board, 
variously  called  as  a bulletin  board,  electronic  board,  and  discussion  board  in  this  study.  In  common,  instructors 
mentioned  that  they  were  using  the  board  as  the  following  three  ways:  discussion,  questions  and  answers,  and  class 
notification.  One  U.S.  instructor  said  how  he  recommended  the  use  of  the  discussion  board  to  students  as  the 
follows: 


. . .1  say,  you  can  post  whatever  you  want  on  the  discussion  board  about  the  mid-term  and  talk  to  each  other 
about  the  mid-term  over  discussion  board,  or  if  they  have  a project  in  groups,  they  use  the  discussion  board 
to  talk  each  other  in  the  group. . . 

As  mentioned  above,  most  of  the  U.S.  instructors  in  this  study  focused  on  students’  discussion  activity  on 
the  web-based  education.  Mainly,  the  discussion  board  can  be  an  appropriate  function  on  the  LMS  to  discuss  in 
asynchronous  discussion.  Interestingly  both  countries’  instructors  said  that  they  are  not  often  using  chatting  function 
as  synchronous  discussion  on  the  web-based  education  because  of  the  difficulty  of  arranging  the  schedule  with 
students.  On  the  other  hand,  Korean  instructors  do  not  offer  a variety  of  approaches  to  usability  of  the  board  on  the 
web-based  education.  Although  they  were  using  the  board  in  their  courses,  they  mentioned  the  limitation  of  the 
usability,  which  is  obvious  in  the  following  statements:  “it  seems  that  the  students  don’t  think  the  board  is 
necessary”.  Similarly  another  instructor  noted,  “students’  reaction  to  the  posted  opinions  is  very  slow”,  and  students 
prefer  to  get  email  rather  than  to  check  out  the  board”. 

With  respect  to  using  email,  we  could  more  hear  from  Korean  instructors  about  why  they  were  trying  to 
use.  Several  reasons  came  out  from  them.  One  was  the  accessibility  issue  on  the  system.  Dr.  I commented  as  the 
follows: 

. . .1  can  say  one  of  the  reasons,  the  accessibility  on  the  system  setting  up  the  web-based  education,  such  as 
they  can  access  from  any  place.  However,  email  is  more  passive  thing  for  students,  on  the  other  hand,  an 
electronic  board  is  more  active  thing  for  students  because  they  have  basically  to  go  the  target  location  of 
board  through  the  log-  in  process  and  read  the  postings  on  the  board.  . . . When  we  sent  email  to  students  or 
they  send  email  to  us,  we  can  get  fast  feedback  from  them.  However,  we  got  very  slow  responses  from 
them  when  we  are  using  on  the  discussion  boards. . . 

In  comparison,  there  were  less  responses  from  the  U.S.  instructors  about  using  email.  Dr.  B mentioned  that, 
“I  asked  them  to  come  together  as  asynchronous  discussion.  Their  group  worked  together  and  solved  the  problem. . . 
It’s  easy  to  manage  the  internal  email.  I asked  them  to  only  use  the  internal  email.” 

As  far  as  using  email,  a different  perspective  between  the  two  countries  was  determined  toward  notification 
functions  of  web-based  education.  The  use  of  function  showed  the  difference.  One  Korean  instructor,  H said,  “I 
think  the  most  important  function  in  the  web-based  education  system  is  the  notification  to  students”.  He  continued 
that  “because  students  have  to  log  in  the  system  to  read  a notice  by  the  instructor,  and  because  they  feel  annoyed  to 
log  in  every  time  to  read  the  notice.”  However,  a U.S.  instructor  emphasized,  “It  is  not  unusual  for  me  to  give 
reminders  students  when  things  are  due”  and  he  explained  the  reason  saying,  “Distance  learning  requires  self- 
directed  engagement.  Students  have  to  structure  for  themselves  sometimes  engaged  in  that  class  turning  in 
materials.” 
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Technology  for  on-and  off-line  instruction 

Based  on  this  finding,  it  is  obvious  that  web-based  education  does  not  need  to  involve  students  who  are  at  a 
geographically  remote  site  to  be  of  benefit  to  instruction  and  learning.  Web-based  resources  can  be  used  in  support 
of  face-to-face  classes  as  well  as  distance  classes.  Lots  of  universities  in  U.S.  and  Korea  are  employing  the  learning 
management  system  such  as  Web-CT.  It  can  be  used  to  make  supplemental  materials  available  on-line  along  with 
off-line  class.  We  could  hear  from  the  Korean  instructors  about  the  LMS  as  mixed  model  for  instruction.  For 
example,  they  were  implemented  that  an  off-line  course  using  the  LMS  for  delivering  course  materials  and  students 
group  activities.  Similarly,  instructors  at  U.S.,  were  using  the  Web-CT  to  create  an  entire  web-based  course  or  to 
supply  materials  available  on  the  web. 

The  LMS  of  institutions  in  both  countries  relied  on  widely  used  commercial  delivery  systems  for  creating 
and  managing  instruction.  In  the  U.S.,  WebCT  or  Blackboard  is  examples  of  such  a system.  On  the  other  hand,  the 
Korean  instructors  mostly  had  support  from  systems  developed  by  Korean  companies.  An  instructor  mentioned  that 
in  Korea  there  was  no  major  company  for  the  LMS.  An  instructor  from  Korea  said  that  although  there  were  the  LMS 
systems  developed  by  the  U.S.  companies  in  Korea,  those  were  not  likely  to  be  selected  by  universities. 

Discussion  and  Conclusion 

This  study  reveals  several  meaningful  findings  in  cross  cultural  web-based  education.  Some  findings  are 
common  on  both  the  US  and  Korean  cultures,  while  others  are  unique  to  each  one’s  own  culture  context. 

First,  the  data  from  this  study  indicated  that  both  countries  instructors  perceive  the  integration  of  the  web- 
based  education  to  be  useful  as  a content  delivery  platform  and  a study  guide.  Both  groups  of  instructors  stated  that 
good  communication  and  well-organized  courses  are  important  in  web-based  education  and  when  the  course  content 
is  well-organized  and  there  is  good  communication  between  and  among  the  learners  and  instructors,  learning 
objectives  and  goals  could  be  reached  by  learners.  According  to  Kanuka  (2001),  “while  good  communication  and 
well-organized  courses  are  important  facets  in  any  learning  environment,  these  factors  are  even  more  important 
when  the  learning  is  at  a distance”  (p.  68).  The  success  of  web-based  distance  education  depends  on  whether  the 
individual  instructors  are  able  to  provide  the  appropriate  opportunity  for,  and  quality  of,  dialogue  between  the 
instructor  and  the  learner,  as  well  as  appropriately  structured  learning  materials  (Moore,  1991). 

Secondly,  in  designing  web-based  instruction,  instructors’  personal  philosophy  and  attitudes  toward  web- 
based  learning,  perceptions  of  learning  needs  from  learners,  and  existing  technical  capacity  to  support  the  web 
delivery  are  determined  to  be  practical  and  critical  factors  implementing  web-based  courses  in  both  countries  college 
education. 

Thirdly,  the  researchers  from  this  present  study  concluded  that  instructors’  perception  toward  web-based 
education  is  affected  by  the  cultural  context  respecting  educational  needs  and  systems.  The  cultural  difference  in 
education  affected  learner’s  learning  patterns  and  needs,  as  well  as  instructional  design  toward  web-based  education. 
According  to  Li  (2000),  the  Western  culture  emphasizes  the  thinking  processes  and  the  learner’s  psychological 
characteristics  such  as  learning  styles,  motivation,  and  intelligence.  Otherwise,  the  Asian  culture  considers  of  the 
learning  as  a process  of  self-perfection  by  seeking  lifelong  commitment,  diligence,  endurance  of  hardship, 
persistence,  and  concentration 

The  researchers  observed  that  the  Korean  distance  educators  consider  that  providing  knowledge  and 
information  to  learners  is  the  most  essential  part  of  learning  in  both  distance  and  face-to-face  classes.  And,  they 
concentrated  more  on  developing  well-organized  Web  materials,  Website,  and  instructional  design  for  learners. 

Also,  the  researchers  found  Korean  students’  learning  needs  are  highly  career  development-oriented  and  strongly 
prefer  acquisition  of  new  knowledge  and  information.  Otherwise,  U.S.  instructors  thought  interaction  is  an  essential 
part  for  learners  to  attain  the  learning  objectives  in  a web-based  distance  environment  as  well  as  face-to-face  classes. 
And,  they  concentrate  more  on  activities  designed  to  promote  student  interaction. 

Recommendation 

Although  this  study  is  limited  to  nine  professors  in  the  U.S.  and  Korea,  the  qualitative  interviews  can  be 
expanded  to  a large  community  of  online  educator  and  learner  to  determine  assessment  of  these  experience, 
perceptions,  and  attitudes  about  teaching  and  designing  web-based  courses  online  through  a web-based  delivery 
platform. 

Knowledge  of  the  distance  education,  understanding  the  potential  of  using  a web-based  course  and  various 
instructional  platforms  for  teaching  in  cross  cultural  contexts,  and  being  aware  of  the  responsibility  involved  in 
teaching  online  course  can  help  facilitate  a meaningful  and  positive  learning  experience  for  educators  and  learners  in 
both  countries.  Future  studies  are  strongly  recommended  to  broaden  the  understanding  of  multi-level  cultural 
contexts  and  their  role  in  web-based  education. 
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Learning  Collaboratively  or  Individually  by  Constructing  Concept  Maps  on 

Computers 
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Description 

Using  a quasi-experimental  design  this  study  investigated  the  comparative  effects  of  individually-generated 
and  collaboratively-generated  computer-based  concept  mapping  on  middle  school  science  concept  learning.  Seventy 
four  students  in  middle  school  science  classes  participated  in  this  controlled  study.  The  findings  revealed  that 
individually-generated  concept  mapping  on  computers  during  study  time  positively  influenced  science  concept 
learning  above  and  beyond  independent  study  and  collaboratively-generated  concept  mapping  on  computers.  In 
addition,  students  had  positive  attitudes  toward  concept  mapping  using  Inspiration  software.  This  study  contributes 
to  the  limited  literature  and  body  of  knowledge  concerning  the  comparative  effectiveness  of  individually  or 
collaboratively-generated  concept  mapping  on  computers. 

Introduction 

Visual  representation  facilitates  both  recall  and  comprehension  (Cifuentes  & Hsieh,  2003a,  2003b;  Gobert 
& Clement,  1999;  Hsieh  & Cifuentes,  2003).  Gobert  and  Clement  (1999)  indicated,  “conceptual  understanding  also 
requires  the  student  to  be  able  to  build  meaningful  and  appropriate  mental  representations,  i.e.,  a mental  model,  of 
the  system  being  taught” (p.  40).  Concept  mapping,  a form  of  visual  representation,  can  be  supported  by  computer 
software  (Chiu,  Wu,  & Huang,  2000;  Fischer,  Bruhn,  Grasel,  & Mandl,  2002).  When  students  have  computer  skills, 
they  can  more  easily  construct,  modify,  or  maintain  their  visualizations  than  they  can  on  paper,  and  skilled  teachers 
can  monitor  and  evaluate  student’s  understandings  effectively  (Chiu,  Wu,  & Huang,  2000).  The  use  of  computer- 
based  visualization  tools  such  as  Inspiration™,  Mind  Mapper™,  and  Microsoft™  Visio  enable  learners  to  interrelate 
the  ideas  that  they  are  studying  in  multidimensional  networks  of  concepts,  and  to  label  and  describe  the  relationships 
between  those  concepts  (Jonassen  et  al,  1998;  Jonassen,  2000). 

According  to  Fischer,  Bruhn,  Grasel,  and  Mandl,  (2002)  processes  of  collaborative  knowledge  construction 
can  support  learners’  scientific  knowledge  construction  more  effectively  than  individual  knowledge  construction. 
Collaboratively-generated  concept  maps  have  power  in  supporting  the  process  of  meaningful  knowledge 
construction.  For  this  reason,  a visual  representation  technique  such  as  concept  mapping  is  often  integrated  into 
collaborative  learning  activities  (Chiu,  Wu,  & Huang,  2000).  Using  concept  maps  allows  learners  to  communicate 
representations  of  their  cognitive  structures  with  other  learners  (Novak  & Go  win,  1996).  According  to  Fischer, 
Bruhn,  Grasel,  and  Mandl,  (2002)  processes  of  collaborative  knowledge  construction  can  support  learners’  scientific 
knowledge  construction  more  effectively  in  the  environment  of  collaborative  versus  individual  learning. 

In  summary,  studies  have  shown  that  concept  mapping  positively  affects  science  concept  learning.  When  students 
have  computer  skills,  computer-generating  concept  maps  positively  affect  learning.  In  addition,  the  literature 
indicates  that  collaboratively-generating  concept  maps  on  paper  or  on  computers  positively  affects  learning.  The 
literature  has  yet  to  report  comparative  effects  of  individually-generating  concept  maps  on  computers  and 
collaboratively-generating  concept  maps  on  computers. 

Methodology 

Using  a quasi-experimental  design,  the  researcher  investigated  the  comparative  effects  of  individually  or 
collaboratively  generating  computer-based  concept  maps  on  middle  school  science  learning.  During  training  in 
concept  mapping,  students  went  through  the  process  of  identifying  three  types  of  text  structure:  sequential, 
categorical,  and  comparative  structures.  Students  then  applied  visual  conventions  to  represent  those  structures  in 
concept  maps.  Students  received  instruction  in  concept  mapping  through  workshop  training,  and  used  their  newly 
acquired  concept  mapping  skills  to  individually  or  collaboratively  generate  computer-based  visual  representations 
during  study  time.  The  hypotheses  were  that  middle  school  students  who  collaboratively  or  individually  generate 
computer-based  concept  maps  would  perform  better  on  a comprehension  tests  than  those  who  do  not  generate 
concept  maps,  and  that  middle  school  students  who  collaboratively  generate  computer-based  concept  maps  perform 
better  on  a comprehension  tests  than  those  who  individually  generate  computer-based  concept  maps. 
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Participants 

Seventy  four  students  in  five  8th  grade  science  classes  at  a middle  school  in  Southeast  Texas  participated  in 
this  study.  The  ethnic  distribution  of  the  classes  combined  was  62%  African  American,  34%  Hispanic,  and  7% 
white.  Ethnicities  were  equally  distributed  across  the  classes.  Over  84%  of  students  were  economically 
disadvantaged.  All  students  and  classes  were  assigned  to  the  same  teacher.  Using  mathematics  and  reading 
performance  scores  on  the  Texas  Assessment  of  Knowledge  and  Skills  to  assure  equivalence  of  student  achievement 
across  groups,  the  researchers  assigned  the  teacher’s  classes  to  one  of  the  three  experimental  groups.  Those  groups 
were-  (a)  control  (group  that  were  not  trained  in  concept  mapping  and  were  told  to  study  using  their  preferred 
strategy),  (b)  students  who  were  trained  to  individually-generate  concept  maps,  and  (c)  students  who  were  trained  to 
collaboratively-generate  concept  maps. 

Design  and  Procedures 

Classes  were  assigned  to  one  of  three  experimental  groups:  control,  individually-generated  concept 
mapping,  and  collaboratively-generated  concept  mapping.  Students  used  the  computer  software  program 
Inspiration™  to  develop  their  concept  maps.  The  dependent  variable  was  a comprehension  test  score  on  science 
concept  learning.  Every  student  had  a computer  account  and  logon  ID  for  the  school  computer  laboratory.  The 
teacher  was  able  to  trace  student  work  and  their  outcomes  on  an  administrator’ s server. 

Both  of  the  experimental  groups  that  created  concept  maps  on  computers  had  attended  three  workshops  on 
computer-based  concept  mapping  in  which  they  learned  how  to  visualize  science  concepts  on  computers  in  their 
computer  laboratory  before  starting  to  study  science  concepts.  These  computer-based  concept  mapping  workshops 
lasted  50  minutes  each  of  three  days.  The  workshops  had  the  same  content  and  materials  for  each  group.  The  science 
content  of  the  workshops  did  not  contain  any  concepts  to  be  covered  during  the  experimental  study  time.  The  control 
group  spent  the  same  amount  of  time  as  the  experimental  groups  watching  a video  about  the  upcoming  science  fair 
as  a placebo  during  workshop  training. 

The  same  teacher  for  all  groups  implemented  instructional  procedures.  When  students  from  any  group 
asked  for  help  and  information,  the  teacher  gave  feedback  equally  to  the  students.  After  three  days  of  the  workshop 
training,  the  control,  individual,  and  collaborative  groups  were  given  the  same  science  essays  to  study  for  five  days. 
The  only  difference  among  groups  was  that  the  control  group  did  not  create  concept  maps  using  computers  and  the 
individual  group  students  worked  independently  to  use  their  learned  concept  mapping  skills  to  create  visual 
representations  that  show  interrelationships  among  concepts  on  computers  during  their  study  time.  However,  the 
collaborative  group  students  were  required  to  study  together  within  small  groups  of  three  using  their  learned  concept 
mapping  skills  to  create  visual  representations  that  showed  interrelationships  among  concepts  on  computers  during 
their  study  time.  Each  of  the  three  group  members  was  assigned  to  be  either  the  group’s  leader,  reporter,  or  monitor. 

The  teacher  gave  10  minutes  of  instruction  for  each  group  in  five  different  science  units  during  each  of  five 
experimental  days.  After  the  teacher’s  instruction,  the  students  in  the  control  group  studied  individually  to  prepare 
for  the  comprehension  test.  The  two  experimental  groups’  students  created  concept  maps  using  computers  for  30 
minutes  and  then  saved  their  files  on  their  computer- server  and  printed  out  their  concept  maps  to  use  during  study 
time.  After  30  minutes  of  study,  each  day  for  five  days,  students  in  each  group  were  given  10  questions  from  the 
computer-based  comprehension  test  to  measure  their  understanding  of  the  science  concepts  in  the  text  that  they 
studied.  A learning  strategy  survey  was  given  to  them  on  the  last  day  to  determine  their  knowledge  of  content  to  be 
tested  prior  to  the  study  and  attitudes  toward  their  experiences  during  the  study. 

The  comprehension  test  consisted  of  40  items  from  the  Prentice  Hall  test  bank  that  was  provided  with  the 
eighth  grade  text  book  adopted  by  the  participating  school  district.  Items  were  criterion  referenced  to  concepts  that 
students  studied  during  their  experimental  study  time. 


Findings 

Using  a quasi-experimental  design,  a one-way  analysis  of  variance  (ANOVA)  was  applied  using  treatment 
group  as  the  independent  variable  and  comprehension  test  scores  as  the  dependent  variable.  A .05  level  was  used  for 
determining  significance.  Levene’s  Test  of  Equality  of  Error  Variances  was  tested  and  found  to  be  homogenous. 
Table  1 shows  that  the  two  experimental  groups’  means  scores  were  respectively  - individual  group  (n=31) 
mean=6.4358  and  collaborative  group  (n=31),  mean=5.9613.  The  mean  score  for  the  control  group  (n=12)  was 
4.9167. 

Table  1 Means  and  Standard  Deviation  on  Comprehension  Posttest  Scores 
Groups  N Mean  Standard 

Deviation 
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Control  Group 

12 

4.9167 

1.77845 

Individual  Groups 

31 

6.4358 

1.23145 

Collaborative  Groups 

31 

5.9613 

1.39668 

Total 

74 

5.9907 

1.47499 

Effect  Size  (control  vs.  individual)  = 1.009 
Effect  Size(control  vs.  collaborative)  = 0.658 


As  shown  in  Table  2,  the  one-way  ANOVA  results  indicated  that  a significant  group  difference  existed 
among  the  control,  individual,  and  collaborative  groups  on  the  mean  scores  of  the  comprehension  posttest,  F =5.118 
(p>  0.05  level). 


Table  2 One  way  ANOVA  Summary  Table  for  the  Effects  of  the  Types  of  Treatment  and  Comprehension  Posttest 
Scores 


Source 

Sum  of  Square 

df 

Mean  Square 

F 

Significance 

Group 

20.011 

2 

10.006 

5.118 

.008* 

Tukey  HSD  post  hoc  test  was  performed  for  group  comparisons.  The  post  hoc  test  result  revealed  that  the 
individual  group  was  significantly  different  than  the  control  group.  This  finding  means  that  the  individual  group 
scored  significantly  higher  than  did  the  control  group  on  the  comprehension  tests  and  that  students  who  individually 
generated  concept  maps  out  performed  students  who  studied  independently  without  being  instructed  to  generate 
concept  maps.  The  effect  size  between  the  control  and  individual  groups  was  1.009.  Therefore,  our  first  hypothesis 
that  middle  school  students  who  collaboratively  or  individually  generate  computer-based  concept  maps  perform 
better  on  a comprehension  tests  than  those  who  do  not  generate  computer-based  concept  maps  is  only  partially 
accepted.  Students  who  collaboratively  generated  concept  maps  did  not  score  significantly  higher  than  did  the 
control  group  on  the  comprehension  test.  Effect  Size  between  the  control  and  the  collaborative  group  was  0.658. 

Also,  there  was  no  significant  difference  between  the  individual  and  the  collaborative  group.  Therefore,  the 
second  hypothesis  that  middle  school  students  who  collaboratively  generate  computer-based  concept  maps  perform 
better  on  a comprehension  tests  than  those  who  individually  generate  computer-based  concept  maps  has  been 
rejected  according  this  Turkey  HSD  post  hoc  test  results. 

Table  3 Post  Hoc  Test  Table  for  the  Effects  of  the  Types  of  Treatment  and  Comprehension  Posttest  Scores 


Tukey  HSD 

(I)  Group 

(J)  Group 

Mean  Difference 

Sig 

Control 

Individual 

1.5191* 

.006* 

Control 

Collaborative 

1.0446 

.079 

Individual 

Collaborative 

.4745 

.380 

P =.05  level 

In  table  4,  the  student  input  on  the  Survey  Result  of  Learning  Strategy  shows  that  attitudes  toward  concept 
mapping  for  science  concept  learning  were  positive.  Creating  concept  maps  using  the  computer  program, 

Inspiration,  provided  students  with  a useful  learning  strategy  and  a positive  experience.  An  interesting  finding  was 
that  the  students  in  the  individual  group  were  more  positively  engaged  in  their  studying  by  themselves  more  than  the 
students  in  the  collaborative  group.  Only  50%  of  the  collaborative  group  students  thought  that  working  with  peers 
was  helpful  and  useful  to  study  science  concept  learning.  According  to  this  interesting  finding,  the  researcher 
assumes  that  students’  negative  attitudes  toward  collaboration  influenced  the  results  and  might  explain  why  the 
individual  group’s  students  performed  better  than  the  collaborative  group’s  students  on  achievement.  When  students 
prepared  for  the  test,  the  control  group  students  just  studied  the  handout  and  read  them,  while  the  two  experimental 
groups’  students  studied  the  relationships  between  bubbles  on  their  concept  maps  and  links  that  they  created  during 
study  time.  They  expressed  that  creating  concept  maps  and  study  relationships  between  bubbles  and  links  were  quite 
helpful,  fun  and  useful  for  learning  science. 


Table  4 Survey  Result  of  Learning  Strategy  during  Study  Time 




Control  Group 

Individual  Group 

Collaborative  Group 

Feeling  about 

None 

87%  of  the  students  say 

87%  of  the  students  say 
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concept  maps 

helpful  and  fun.  Some  students 
think  better  than  worksheet  on 
paper  or  regular  class  activity 

helpful  and  fun.  Some  students 
think  better  than  worksheet  on 
paper  or  regular  class  activity 

Helping  to  learn 
the  content 

None 

87%  of  the  students  think 
concept  maps  help  with 
learning  the  science  content. 

97%  of  the  students  think 
concept  maps  help  with 
learning  the  science  content. 

Opinion  of 
working  with 
group 

None 

None 

50%  of  the  students  think  that 
working  with  group  is  helpful 
and  useful.  40  % students 
think  negative.  10  % No 
response 

Opinion  of 
working  with 
individual 

None 

70%  of  the  students  think  that 
studying  by  themselves  is 
helpful  and  useful.  19% 
students  think  negative.  1 1 % 
No  response 

None 

Steps  of  the 
preparing  the 
test 

Students  read  the  whole 
handout  and  some 
students  take  notes  and 
underline  parts. 

Students  studied  their  concept 
maps  and  tried  to  understand 
relationship  among  the 
bubbles 

Students  studied  their  concept 
maps  and  try  to  understand 
relationship  among  the 
bubbles 

Exposure  of 
concept  maps 
before 

None 

12%  students  knew  the 
concept  mapping  before. 

9%  students  knew  the  concept 
mapping  before. 

Discussion  and  Conclusions 

The  findings  of  this  study  revealed  that  individually-generating  concept  maps  during  study  time  positively 
influenced  science  concept  learning,  at  least  for  eighth  graders.  However,  the  findings  do  not  support  Fischer,  et  al’s 
(2002)  assumption  that  collaborative  knowledge  construction  is  more  effective  than  individual  knowledge 
construction.  Even  though  there  was  no  significant  difference  between  individual  and  collaborative  groups  in  this 
group  of  participants,  the  researchers  who  conducted  this  study  do  not  conclude  that  one  approach  is  universally 
superior  to  another.  In  this  case,  the  reason  that  the  collaborative  group  did  not  score  significantly  higher  than  the 
control  group  on  achievement  might  have  been  lack  of  a disciplined,  supportive  collaborative  working  environment 
and  lack  of  experience  with  collaborative  processes.  Students  in  the  collaborative  groups  spent  excessive  time 
competing  the  keyboard  and  were  generally  distracted  by  their  group  members.  Most  of  the  participants  did  not 
have  computers  at  home  and  learned  concept  mapping  for  the  first  time  in  the  context  of  this  study.  In  addition,  the 
school  district  had  limited  technical  facilities.  Also,  most  students  had  had  few  opportunities  to  develop 
collaborative  learning  skills  in  their  young  school  careers.  In  another  school  context,  where  students  have  technical 
skills  and  support,  the  atmosphere  is  conducive  to  collaboration,  and  students  have  a history  of  collaborative 
experience  in  school,  the  finding  may  be  more  positive  for  collaboratively-generated  concept-mapping.  Researchers 
might  investigate  whether  individually  or  collaboratively  computer-generated  concept  maps  differentially  effect 
learners  with  specific  characteristics  such  as  those  listed  above.  Given  that  participants  in  this  study  were  distracted 
by  members  of  their  group,  both  the  classroom  teacher  of  those  students  and  the  researchers  think  that  an 
investigation  comparing  collaborative  groups  versus  collaborative  pairs  is  of  interest.  This  study  contributes  to  the 
limited  literature  and  body  of  knowledge  concerning  the  effectiveness  of  individually  versus  collaboratively- 
generating  concept  maps  on  computers. 
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Abstract 

This  is  the  study  of  development  of  media  education  curriculum  in  the  elementary  schools.  Supporting  the 
new  concept  of  the  digital  prosumer's  performance,  the  project-based  learning  is  considered  as  one  of  effective 
instruction  theories.  Following  this  idea,  this  media  education  curriculum  is  developed  for  the  6-9  ages  students. 
This  paper  also  includes  the  pictures  of  the  program. 

Introduction 

The  necessity  of  media  literacy  has  been  discussed  since  this  information-rich  society,  to  keep  pace  with 
the  knowledge  and  technological  expertise  necessary  for  finding,  applying,  and  evaluating  information.  Today  the 
research  fields  for  media  education  are  getting  voluminous  with  change  the  concept  for  media  education 
environment  as  to  ‘digital  environment.’  This  digital  environment  has  three  key  meanings  for  media  education.  First 
it  is  easy  for  students  to  express  the  meaning  with  the  various  symbol  systems.  Second  the  hardware  and  software, 
which  is  to  shape  the  media  text,  are  developing  everyday  for  the  efficient  use.  Last  key  point  is  the  raising  issues  of 
developing  theories  and  the  tools-  which  is  to  share  the  digital  media  text. 

This  digital  environment  has  brought  out  the  new  concept  ‘ the  digital  prosumer .’  Digital  prosumer  is 
referring  to  the  person  who  accepts  media  information  critically  as  well  as  engages  actively  in  production  of  media 
information.  In  other  words,  ‘ the  digital  prosumer ’ is  the  combination  of  ideas:  media  text  producers  and  the  media 
text  receptors(Lim,  2003).  For  enlarging  the  digital  prosumer’s  performance  area18  the  media  education,  certainly,  is 
required  with  the  great  support  of  the  digital  media  environment. 

As  the  concept  of  digital  prosumer  and  its  performance  area  have  just  introduced,  the  concrete  instructional 
strategies  for  developing  the  media  environment  and  media  curriculum,  are  not  full  enough  made  out.  To  cover  this 
limitation  for  developing  the  media  curriculum  for  enlarging  digital  prosumer’ s performance,  consequently,  this 
study  has  taken  up  the  most  suitable  instructional  theory,  project-based  learning,  as  just  the  sub-instructional 
strategies. 

Therefore,  the  purpose  of  this  study  is  to  develop  media  education  curriculum  for  enlarging  the  digital 
prosumer’s  performance  area.  The  main  design  principle  is  the  contents  design  strategies  of  media  literacy 
education  to  cultivate  the  digital  prosumer  with  the  sub-support  of  project-based  learning  strategies. 

Theoretical  Framework 

Digital  Prosumers’  Performance  Area 

Kwon  and  Sim(2005)  suggested  the  designing  strategies  of  media  literacy  education  to  enlarge  the  digital 
prosumers'  performance  area.  Kwon  and  Sim  illustrated  the  three  axis  related  to  the  digital  consumers'  performance 
area.  They  are  'multiplicity  of  symbol  systems',  'acceptance  and  production  of  media  text',  and  'sharing  of  media 
text'. 

'Multiplicity  of  symbol  systems'  is  concerned  with  how  to  use  diversely  the  symbol  systems.  Learners  need 
to  know  about  the  effective  way  of  communication  by  understanding  the  characteristics  of  each  symbol  system. 
They  also  need  to  understand  the  principals  of  effective  coordination  among  various  symbol  systems. 

'Acceptance  and  production  of  media  text'  is  related  to  whether  learners  only  accept  the  information,  or 
whether  not  only  accept  the  information  but  also  produce  the  media  text  using  the  information  they  accepted.  In 
relation  to  acceptance,  learners  need  to  actively  access  the  media  text,  then,  accept  the  media  text  critically  and 
selectively.  In  relation  to  the  production  of  media  text,  learners  need  to  acquire  the  two  kinds  of  competencies.  One 
is  associated  to  the  data  creation,  transformation,  edition,  and  publishment  of  the  media  text.  The  other  is  related  to 
the  critical,  creative  and  complex  thinking  in  regarding  the  media  text. 


18  Detail  explanation  for  the  digital  prosumer’s  performance  area  is  introduced  in 
theoretical  backgrounds. 
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'Sharing  of  media  text'  is  associated  with  whether  learners  personally  possess  their  media  text  or  openly 
share  with  others.  When  they  share  the  media  text,  learners  perform  the  two  roles,  consumer  and  producer.  Learners 
as  consumers  need  to  critically  understand  and  interpretate  toward  the  producers  and  the  products(The  New  London 
Group,  1995).  Learners  as  producers  need  to  have  the  attitude  that  adjustably  accommodate  the  consumers  opinion. 
Every  learners  should  be  familiar  with  the  tools  and  spaces  for  sharing  media  texts. 


sharing  media 
texlsfy) 


Figure  l.Tthree  axis  of  the  digital  consumers'  performance  area 


Project-based  Learning 

This  study  applies  the  project-based  learning  as  the  sun  instructional  strategies.  So  instead  of  detail 
information  on  the  project-based  learning,  the  concept  and  the  instructional  method  for  PBL  is  shortly  presented  in 
this  article. 

Project-based  learning,  based  on  the  constructivism,  creates  opportunities  for  students  to  work  on  problems  in  the 
real  situation.  Teachers  bring  projects  relevant  to  the  real  problems,  coming  out  outside  the  classroom,  and  help  the 
students  to  see  and  understand  the  connections  between  classroom  activities  and  the  real  situation  of  the  work.  These 
projects  can  also  build  students  abilities  to  set  group  goals  with  collaboration  among  peers.  The  interdisciplinary 
nature  of  these  projects  encourages  students  to  widen  and  explore  their  personal  interests  while  gaining  the 
knowledge  they  need  in  core  subjects(Leffy,  1998). 

Three  main  principles  for  project-based  learning  are  as  the  following(Penuel  et.  al,  1999).  First  key 
principle  is  the  student’s  full  participation,  leading  his/her  project.  Students  get  the  chance  for  making  the  decision 
of  all  the  problems  when  working  the  project.  And  this  leads  students  to  get  responsibility  for  their  work  and  their 
peers.  Secondly,  students  learn  how  to  present  all  the  process  of  research  activities  and  the  final  results  with 
effective  use  of  the  media  literacy.  Last  principle  brings  learners  to  have  broader  ideas  for  understanding  and 
developing  the  curriculum.  Both  learners  and  teachers  are  making  the  curriculum  together,  so  the  curriculum  can  not 
be  fixed  before  the  class  starts. 

Six  strategies  for  PBL  are  summarized  as  followings. 

First , scaffolding  is  used  for  helping  students  who  lose  the  ways  to  work  the  project.  Second,  coaching  will 
lead  students  to  finish  their  job  on  the  level  of  experts.  In  other  words,  the  project  quality  will  be  advanced  up  to  the 
project  done  by  experts.  Third,  ample  resources  for  projects  are  concretely  provided  for  supporting  for  all  the 
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projects.  Forth,  each  different  knowledge  representation  will  contribute  to  the  activities  for  collecting,  selecting, 
summarizing,  and  creating  knowledge.  Fifth,  the  process  of  collaboration,  and  sharing  ideas,  and  feedback  will 
proceed  naturally  to  encourage  the  communication.  Last  strategy  is  providing  activity  for  reflection.  This  activity 
will  be  great  help  for  evaluating  the  project  process  and  the  results,  and  personal  participation. 


Design  and  Development 

This  study  develops  the  media  literacy  curriculum,  ‘Building  the  Garden  of  Media’  for  the  l~3grades.  The 
main  design  principle  is  the  contents  design  strategies  of  media  literacy  education  to  cultivate  the  digital  prosumer 
with  the  sub- support  of  project-based  learning  strategies. 


Module 

Flow 


Module 

Theme 


hr. 


output 


Video 


Learning  Materials 
Pictures 


Prologue 

What  is  the  media? 

1/1 

o 

o 

Media  and  I 

My  face  in  the  pond 

2/4 

o 

Making  Media  Tree 

2/4 

o 

Making  Media 

Making  Story  of  Media 
Garden 

4/11 

o 

Media  Train 

4/11 

o 

o 

Finding  the  music 

3/11 

o 

Understanding 

Media 

TV! 

3/6 

o 

o 

Recognizing  the  Media 
Garden 

3/6 

o 

Epilogue 

Displaying  the  final 

3/3 

o 

Table  1.  Strategies  of  media  literacy  education 
to  enlarge  the  digital  prosumers'  performance  area 


Music 


O 


Table  1 shows  the  strategies  of  media  literacy  curriculum  based  on  the  concept  of  three  axes  related  to  digital 
prosumers'  performance  area. 


No. 

Axis 

Educational  Contents 

Teaching  and  Learning  Strategies 

1 

multiplicity  of  symbol 
system 

■ understand  the  unique 
characteristics  of  each  symbol 
system  and  practice  their 
utilization 

■ understand  multiplicity  of 
various  symbol  systems  and 
practice  its  utilization 

■ understand  the  characteristics 
of  symbol  systems  used  in  each 
media 

■ guiding  for  learning  activity 

■ offering  good  & bad  cases  for  use  of 
symbol  system 

■ knowledge  on  grammar  of  each  symbol 
system 

2 

acceptance  and 
production  of  media 
texts 

■ critical  understand  and 
interpretation  of  media  text 

■ practice  the  skills  needed  for 
the  media  text  production 

■ creative  production  of  the 
media  text 

■ facilitating  practice  for  critical  thinking 

■ offering  the  evaluation  criteria  for  project 
outcome 

■ guiding  the  process  of  project  activity 

■ offering  tools  for  group  activity  support 

■ offering  online  spaces  for  group  activity 
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support 

3 

sharing  media  texts 

■ attitude  of  consumer  who 
shares  media  text 

■ attitude  of  producer  who 
shares  media  text 

■ practice  the  skills  needed  to 
use  tools  and  spaces  of  media 
text  sharing 

■ suggesting  the  desirable  attitude  as 
consumers/producers 

■ offering  the  tools  for  sharing  media  text 

■ offering  the  tools  for  (a) synchronous 
discussion 

■ offering  the  tools  for  reflection 

■ guiding  how  to  use  the  tools  and  spaces 
for  sharing 

Table  1.  Strategies  of  media  literacy  education 
to  enlarge  the  digital  prosumers'  performance  area 


The  following  strategies  are  more  precisely  for  contents  design  on  the  basis  of  project-based  learning(Kwon,  Seo. 
Kang,  2002). 


No. 


1 

2 

3 

4 


5 

6 


Characteristics 


Educational  Content 


Help  students  who  lose  the  ways 
to  work  the  project. 

Lead  students  to  finish  the  project, 
advancing  to  the  one  by  experts. 

Full  resources  for  projects, 
provided  for  supporting  for  all  the 
projects 

Each  different  knowledge 
representation,  contributing  to  the 
activities  for  collecting,  selecting, 
summarizing,  and  creating 
knowledge. 

the  process  of  collaboration,  and 
sharing  ideas,  and  feedback,  to 
encourage  the  communication. 

Providing  activity  for  reflection 
helping  for  evaluating  the  project 
process  and  the  results,  and 
personal  participation. 


Teachers  and  peers,  supporting 
students  to  lead  the  project 

Teachers’  advice, , bring  synergy 
effects  among  peers  and  the 
student 

Resources,  guiding  students  to 
find  the  project  themes  in  real  life 
situation 

Use  of  multiple  media,  shaping 
the  learning  contents,  and  materials 


Interactions,  raising  through  co- 
working of  the  project  with  peers. 


■Discussions  with  teacher 
and  the  students,  and  peers, 
sharing  and  developing  ideas 
■Reflect  on  student- 
oriented  learning  process,  not 
teacher-oriented  and  evaluate  it 
Table  2.  Strategies  of  media  literacy  education 
based  on  the  project-based  learning 


Teaching  and  Learning 
Strategies 

Scaffolding 

Coaching 


Resource  support 


Knowledge  representation 


Communication  and 
collaboration 

Reflection 


Results 

Media  curriculum,  ‘Building  the  Garden  of  Media’,  has  divided  its  module  flow  into  three,  ‘Media  and  I’, 
‘Making  Media’,  and  ‘Understanding  Media.’  And  the  three  main  parts,  including  prologue  and  epilogue,  has  nine 
themes.  ‘Building  the  Garden  of  Media’  has  published  into  three  types:  a book  for  teachers  or  facilitators,  a 
workbook,  and  CD  including  all  the  learning  materials.  In  CD,  pictures,  audio  materials,  and  movies  are  contained. 
Videotape  for  movies  was  provided,  either. 

‘Building  the  Garden  of  Media’  is  developed  for  the  six-nine  ages,  containing  the  contents  on 
understanding  the  media,  and  its  principles  for  its  effective  use,  and  critical  analysis  skills  on  media  education.  This 
program  is  to  lead  students  to  identify  and  solve  the  project  problems  for  themselves  - which  is  the  main  purpose  of 
the  PBL,  project-based  learning.  Besides,  the  levels  of  contents  are  introduced  for  the  differences  in  cognitive 
development,  and  the  collaboration  work  is  encouraged  with  systematic  design. 


Module  Module  Theme  hr. 


Learning  Materials 
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Flow 

Video 

Pictures  Music 

Prologue 

What  is  the  media? 

1/1 

O 

O 

Media  and  I 

My  face  in  the  pond 

2/4 

O 

Making  Media  Tree 

2/4 

O 

Making  Media 

Making  Story  of  Media 

4/11 

O 

Garden 

Media  Train 

4/11 

O 

O 

Finding  the  music 

3/11 

O 

O 

Understanding 

TV! 

3/6 

O 

O 

Media 

Recognizing  the  Media 

3/6 

O 

Garden 

Epilogue 

Displaying  the  final  output 

3/3 

O 

Table  3.  ‘ Building  the  Garden  of  Media  ’ module  matrix 


This  program  has  six  advantages  for  effective  use  for  media  curriculum. 

First,  topic  ‘building  the  garden  of  media’  leads  students  be  not  just  the  receivers  of  the  media  text,  but  the 
prosumer  in  the  media  environment.  Besides,  the  name  ‘Building  the  garden’  takes  the  images  of  making  their  own 
ideas  on  media  for  themselves. 

Second,  students’  ages  are  reckoned  when  designing  the  media  curriculum.  Various  learning  materials  and 
resources  are  designed  according  to  the  ages,  for  helping  students  do  the  work. 

Third,  the  project  is  the  main  activity  in  this  curriculum.  Through  the  project,  students  get  the  concept  or 
principles  for  understanding  the  media  text,  and  the  digital  prosumer. 

Forth,  the  program  schedule  is  not  strongly  fixed.  So  teachers  change  the  schedule  easily,  depending  on  the 
class  situation  such  as  the  unexpected  periods  that  learners  take  to  finish  one  lesson.  Additionally,  this  curriculum 
can  also  be  actively  used  in  the  workshop  or  seminars. 

Fifth,  the  absolute  media  preparation  is  not  required  in  this  project.  Various  methods  for  active 
communication  is  introduced  instead  of  using  the  cutting  edge  media. 

This  curriculum  was  done  through  the  workshop  in  June  2001.  Following  table  is  the  feedback  from  the  interview 
with  teachers  after  implementing  this  program19. 


No.  Three  Key  Point 


Feedback  in  details 


1 


2 

3 


Teacher’s  role 


Time  Schedule 


Contents 


■ Making  students  have  interest  in  lessons  when  beginning  the  class(This 
have  brought  out  the  project  quality  difference) 

■ Not  goal  oriented,  but  process-oriented  class  is  needed. 

■ Leading  active  need  communication  and  solving  the  conflicts  or  fights 
among  students  are  necessary 

■ Patience  is  required  to  make  students  lead  the  project 

■ To  keep  attention  on  students’  process  of  learning  lessons  is  required  for 
time  schedule  change.  Usually  more  time  is  needed  comparing  to  the  time 
plan  in  the  workshop. 

■ This  curriculum  is  absolutely  one  of  alternative  solutions  for  having 
students  with  full  participation 


19  In  June  2001,  a workshop  was  done  for  4days  3nights.  Teachers  and  students  in  various  public  schools  have 
participated  in  here.  Interview  for  evaluating  the  curriculum  was  done  after  the  workshop.  The  table  is 
summarized  shortly. 
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■ Students’  understanding  of  concepts  are  not  obviously  represented  easily. 

But  the  project  process  and  its  output  clearly  show  their  understanding  of 
the  lessons. 

Table  4.  Teacher’s  Feedback  after  ‘ Building  the  Garden  of  Media’  workshop 

Conclusion 

This  study  suggests  four  new  ideas  for  developing  the  media  education  in  Korea. 

First,  both  theorists  and  practitioners  should  design  the  effective  curriculum  of  instruction,  with  organizing 
various  groups  of  learners  and  teachers. 

Second, , the  new  paradigm  of  the  instruction  in  the  field  of  the  open  media  education  should  be  studied 
using  the  media  tools  with  the  enthusiasm  of  improving  the  instruction. 

Third,  When  designing  the  specific  media  literacy  education  program,  to  enlarge  the  digital  prosumers' 
performance  area,  the  designer  needs  to  develop  the  program  more  elaborately  applying  the  systemic  instructional 
design  process  after  deciding  the  target  audience  and  the  project  content. 

Fourth,  The  roles  of  educational  technology  needed  for  future  desirable  media  literacy  education  is 
suggested. 
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Creating  Video  Products  to  Serve  Educational  and  Non-Profit  Organizations: 
A Community-Based  Experiential  Learning  Experience  for  Students 

Feng-Qi  Lai 
Indiana  State  University 

Abstract 

This  paper  introduces  an  interactive  video  production  class.  In  this  class , the  learning  activities  were 
designed  and  structured  to  facilitate  student  application  of  classroom  learning  to  a community-based  setting.  Based 
on  information  gathered,  the  author  summarizes  and  compares  students'  reflection  and  learning  outcomes  in  the 
two  semesters.  The  author  also  provides  suggestions  in  terms  of  design  and  implementation  of  community -based 
experiential  learning. 


Introduction 

Experiential  learning  is  rooted  in  doing  and  experiencing  (Hutton,  1989).  An  interactive  video  class  usually 
requires  student  to  do  a project  or  several  smaller  projects  to  learn  camera  operation  and  video  editing  skills. 
However,  being  able  to  operate  a camera  and  edit  video  clips  does  not  equal  ensuring  students  to  be  able  to  meet  the 
needs  that  the  workforce  requires.  Projects  created  by  educational  technology  students  are  more  meaningful  when 
the  projects  serve  educational  purposes.  For  example,  an  interactive  video  of  an  instructor  explaining  difficult  points 
of  a lesson  can  be  very  helpful  for  an  online  course.  For  another  example,  an  interactive  video  of  drills  and  practice 
for  mathematics  remediation  purposes  can  be  used  for  as  many  classes  as  needed  without  requiring  teachers’  extra 
work  hours.  These  projects  serve  educational  purposes  and  make  students’  use  of  time  doing  projects  more 
meaningful  for  the  class.  Based  on  its  nature,  the  interactive  video  class  can  be  designed  to  facilitate  student’s 
application  of  classroom  learning  to  a community-based  setting,  so  that  the  projects  that  students  have  created  can 
be  used  by  schools  or  non-profit  organizations  to  help  with  teaching,  research,  and  service.  According  to  Boy  dell 
(1976),  experiential  learning  is  synonymous  with  meaningful-discovery  learning.  According  to  Moon  (2004,  p.  116), 
“we  are  not  likely  to  learn  from  experience  unless  there  is  some  intention  or  will  to  learn.”  Engaged  in  the 
community,  through  experiential  learning,  students  are  expected  to  be  able  to  learn  more  meaningfully  and  their 
critical  thinking,  problem  solving,  technical,  and  communicational  skills  that  the  workforce  requires  are  expected  to 
be  enhanced. 

The  discussion  of  this  paper  is  based  on  two  semesters’  information,  one  in  a 16-week  fall  semester  and  one 
in  a five-week  summer  session. 


Design  of  the  Class  Activity 

This  interactive  video  class  was  designed  using  a community-based  experiential  learning  approach  to  serve 
two  purposes:  1)  to  prepare  students  for  working  in  a real  world  situation  and  2)  to  create  projects  that  help  the 
community  with  their  teaching,  research,  and/or  services.  Students  were  told  to  do  the  project  in  a group  of  two  to 
three.  Projects  needed  to  meet  the  needs  of  the  community  partners  for  educational  / instructional  purposes.  The 
finalized  projects  needed  to  include  a title,  text,  graphics,  background  music,  narrative,  and  special  effects  where 
appropriate.  Each  project  should  take  approximately  around  10  minutes  to  view,  focusing  on  one  specific  topic. 

Role  of  students  in  project  development 

1 . Contacting  a school,  a department  at  the  university,  or  a non-profit  organization  (using  a questionnaire)  to  find 
out  the  need  for  the  project. 

2.  Defining  the  project  and  contacting  the  community  partner  to  be  involved  to  confirm  that  the  plan  was  feasible 
and  the  activities  would  be  supported. 

3.  Writing  a proposal  of  the  project  describing  the  proposed  topic  and  stating  rationale  and  the  confidence  to 
complete  the  proposed  project. 

4.  Upon  the  approval  of  the  proposal  by  the  instructor,  preparing  the  production  planning  document,  including 
goal  statement,  target  audience,  communication  objectives,  content  outline,  creative  treatment  for  the  product 
(background  music,  narrative,  transitional  effects  etc.),  shot  breakdown  sheet  (shot  description  and  location), 
script  (video  and  audio  for  each  shot),  and  projected  timeline.  The  project  to  be  created  should  meet  the 
community  partner’s  goal  and  the  needs  of  the  target  audience. 

5.  Video  shooting,  editing,  and  compressing. 

6.  Presenting  the  project  in  class,  debriefing,  and  peer  evaluation. 
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7.  Revising  (if  needed). 

8.  Sending  the  project  to  the  community  partner  for  preview. 

Role  of  community  partner(s)  in  project  development 

1 . Receiving  students  and  discussing  the  project  with  students  to  help  define  a topic. 

2.  Providing  content  information  for  the  video  product. 

3.  Scheduling  videotaping  sessions  with  students. 

4.  Reviewing  the  draft  video  product  and  providing  feedback. 

Role  of  the  instructor 

• Guiding  and  supervising  students’  learning  activities 

• Ensuring  quality  of  students’  projects. 

Student  Profile  and  Work  Groups  in  the  Two  Semesters 

There  were  five  students  enrolled  in  the  fall  semester,  one  American  and  four  international  students.  They 
formed  two  groups.  Students  themselves  decided  which  group  to  join.  After  talking  to  one  another,  two  international 
students  formed  one  group  and  the  other  three  formed  a second  group.  The  two- student  group  decided  to  work  with 
the  International  Affairs  Center  and  create  a project  that  would  introduce  every  aspect  of  the  university  that  an 
international  student  needed  to  know  before  they  came  to  the  US.  The  other  group  did  not  have  a topic  until  they 
went  to  talk  to  the  community.  They  finally  determined  a topic  that  they  thought  was  meaningful.  They  decided  to 
work  on  a project  for  a senior  home. 

All  five  students  in  the  fall  class  were  highly  motivated.  They  scheduled  their  meetings,  discussing  their 
plans,  and  working  long  hours  together.  Their  peer  evaluation  and  reflection  were  very  positive  (details  will  be 
discussed  in  the  “Student  peer-evaluation  and  reflection”  section). 

In  summer,  six  students  were  enrolled,  two  school  teachers,  one  local  librarian,  one  university  staff,  and 
two  international  students.  One  of  the  school  teachers  requested  to  do  an  individual  project  for  his  school  and  the 
university  staff  member  also  requested  to  do  an  individual  project  for  her  own  university.  The  two  international 
students  formed  one  group  and  the  remaining  two,  the  librarian  and  the  other  school  teacher  who  was  enrolled  for 
teaching  license  renewal  formed  another  group.  The  first  group  was  interested  in  introducing  an  English  language 
program  to  international  students  and  the  second  group  was  discussing  teaching  Spanish  as  the  project  topic.  The 
two  students  in  the  second  group  had  schedule  conflicts  because  the  school  teacher  was  living  about  70  miles  away 
from  the  campus.  Later,  this  school  teacher  dropped  the  class.  As  a result,  there  was  only  one  group  project  and 
three  were  working  on  their  individual  projects. 

Students  in  the  summer  session  were  very  excited  about  the  project  at  the  beginning  of  the  class.  However, 
towards  the  end  of  the  semester,  the  three  students  who  were  doing  individual  projects  were  frustrated.  They 
expressed  their  wishes  that  they  should  have  had  a team  worker.  They  also  said  because  they  did  not  have  a good 
plan  at  the  beginning,  they  were  not  able  to  fulfill  something  that  they  had  thought  was  easy  to  do. 

Learning  Outcomes 

Learning  outcomes  of  this  class  were  evaluated  in  three  aspects:  the  video  project,  peer-evaluation  and 
reflection,  and  presentation  and  professional  critique.  They  are  discussed  in  the  following  sub-sections. 

Video  projects 

The  projects  that  were  created  by  the  fall  students  were  of  comparatively  higher  quality  concerning  both 
technical  skills  and  the  content.  The  project  introducing  the  university  to  international  students  started  with  welcome 
remarks  from  an  interim  associate  director  of  the  International  Affairs  Center,  which  was  followed  by  video  shots  of 
every  detail  about  living  on  campus  that  an  international  student  needed  to  know.  The  group  interviewed  the 
International  Affairs  Center  to  gather  all  information  needed  for  the  project,  including  official  documents,  academic 
life,  financial  help,  living  conditions,  up  to  the  homecoming  and  the  international  festival.  The  video  clips  of  all 
information  were  efficiently  and  appealingly  edited  together.  The  background  music  was  exciting.  The  narration  was 
spoken  using  an  international  student’s  voice,  which  made  the  video  even  more  like  home  to  international  students. 
The  video  project  ended  with  the  interim  associate  director’s  welcome  remarks  together  with  the  contact 
information.  The  International  Affairs  Center  was  happy  with  the  project. 

The  second  group  created  a more  professionally-looking  project.  The  three  students  of  the  group,  in  their 
own  words,  were  perfectionists.  They  did  video  taping  and  editing  together.  For  every  scene,  they  videotaped  from 
three  different  angles,  so  that  they  were  able  to  compare  the  video  footages  to  decide  which  one  was  the  best  for  the 
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editing  of  the  project.  The  purpose  of  the  project  is  to  help  senior  citizens  find  a home.  The  content  covered  all 
facilities  available  for  senior  citizens,  including  the  library,  the  bank,  the  media  room,  the  clinic,  the  dinning  room, 
the  physical  fitness  facilities,  up  to  various  services  such  as  cleaning,  transportation,  etc.  For  each  section,  the  video 
showed  viewers  what  forms  needed  to  be  filled  out  for  living  in  this  senior  home.  The  students  used  a unique 
technique  of  transferring  static  pictures  from  black  and  white  to  sepia  and  then  to  color  instead  of  traditional 
transitional-effects  when  editing  the  video  clips.  The  transition  from  static  pictures  to  dynamic  motion  made  the 
whole  project  much  more  vivid.  The  background  music  was  soft  and  graceful.  The  location  of  text  was  also 
carefully  considered.  Students  were  very  proud  of  their  project.  Although  they  spent  long  hours  on  the  project,  they 
thought  the  project  was  worth  their  time  because  they  were  rewarded  with  not  just  knowledge  and  skills  but  also 
self-satisfaction. 

In  general,  the  projects  created  by  the  summer  students  were  not  as  of  high  quality  as  those  created  by  the 
fall  students.  Both  video  shots  and  editing  were  rougher.  The  organization  of  the  three  individual  projects  was  not  as 
good  as  the  project  that  was  created  by  the  group.  Students  said  that  they  had  expected  a better  project  than  what 
they  had.  They  concluded  three  major  reasons:  1)  five  weeks’  time  was  too  short  for  doing  a project  at  that  scope;  2) 
during  the  summer  time,  it  was  not  as  easy  for  them  to  get  community’s  cooperation;  and  3)  they  (the  three  students 
who  did  individual  projects)  should  have  had  a team  worker  working  on  the  project. 

Student  peer-evaluation  and  reflection 

Every  student  was  required  to  do  self-reflection  and  students  who  did  a group  project  were  required  to  write 
a peer  evaluation  on  the  team  work.  Most  members  of  the  three  groups,  two  in  fall  and  one  in  summer,  evaluated 
their  peers’  work  and  attitude  highly.  They  expressed  their  appreciation  for  the  team  work.  They  pulled  their  minds 
together  and  learned  from  each  other’s  strength.  They  said  that  without  their  partner(s)’  contribution,  the  project 
would  never  be  so  professional  and  their  work  experience  would  not  be  so  satisfactory.  The  key  words  that 
expressed  their  appreciation  about  their  partners’  contribution  include  the  following: 

• “I  really  enjoyed  working  on  this  project  with  my  partner.” 

• “He  had  very  sharp  observation  - he  pointed  out  many  problems  I could  have  passed  while  we  were  editing  the 
video  and  audio...” 

• “I  would  say  we  were  meant  to  be  a group  together  because  we  could  fill  up  the  weakness  for  each  other.” 

• “He  is  willing  to  try  new  things. . . Because  of  his  willingness  of  trying  new  things  on  our  video,  we  were  able  to 
use  many  different  effects  on  our  video.” 

• “He  is  also  a good  team  player.” 

• “I  had  a very  good  experience  working  with  my  partner  and  he  made  a great  contribution  to  this  team.  I look 
forward  to  work[ing]  with  him  again.” 

• “My  partner  is  an  intelligent  and  hard-working  worker.  I am  so  impressed  by  her  creativity.  Her  idea  usually 
inspired  me  a lot.” 

• “She  is  a dedicated  professional  that  attends  each  meeting  with  a spirit  of  collaboration  and  cooperation.” 

• “She  is  a hard  worker,  a perfectionist  and  a very  dependable  person.” 

• “She  has  a great  eye  for  details  and  uses  appropriate  technical  vocabulary  to  describe  the  elements  and  points  of 
improvement  in  our  project.” 

• “Her  guidance  during  the  shots  as  well  as  the  interaction  with  the  personnel  and  residents  [in  the  community] 
created  the  atmosphere  and  the  way  to  success  for  our  project.” 

• “His  technical  skills  became  evident  especially  during  the  editing  process  and  in  making  comments  regarding 
the  project.” 

• “His  dedication  and  dependability  were  greatly  appreciated.” 

• “During  the  shots,  he  conducted  himself  with  the  expected  professionalism,  working  collaboratively  in  a helpful 
way.” 

• “He  is  obviously  a dedicated  individual.” 

• “Both  my  partners  are  high  achievers  and  perfectionists  who  push  themselves  and  the  team  to  a better  standard 
than  usual.” 

• “He  has  a very  keen  eye  for  the  artistic  temperament  of  a picture,  movie  or  sound  and  she  is  very  observant 
about  the  nuances  of  a clip  while  keeping  the  big  picture  in  focus.” 

• “Repetitive  tasks  were  speeded  up  by  two  or  all  three  persons  manning  the  keyboard  and  mouse.” 

• “He  and  she  also  improved  on  each  other’s  ideas,  improving  the  quality  of  the  project  and  in  my  opinion 
surpassing  the  expectations  of  quality  we  had  set  for  ourselves.” 
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• “I  believe  as  teammates  we  had  a very  symbiotic  relationship  and  I would  work  with  them  again  given  the 
opportunity.” 

• “He  was  always  willing  to  try  something  new  even  if  meant  redoing  a previous  clip.” 

• “I  appreciated  their  energy  and  patience  with  this  project.” 

• “Many  times  we  were  staying  and  working  on  the  project  until  the  early  hours  of  a new  day.  All  of  us  are 
perfectionists  and  wanted  to  create  the  best  project  possible.” 

• “We  knew  the  parameters  and  criteria  of  excellence  expected  of  this  project.  I believe  this  was  accomplished 
with  their  insight,  input,  and  skill.” 

One  student  was  not  very  satisfied  with  her  partner’s  performance.  She  complained  about  her  partner’s  lack 
of  computer  experience.  Her  partner  was  a new  international  student.  This  international  student  said  in  her  peer- 
evaluation  that  her  partner  dominated  editing  so  she  did  not  have  much  chance  of  improving  her  technical  skills. 
However,  they  completed  the  project  together  and  both  commented  that  it  was  a great  experience  for  them  to  work 
with  their  partner  in  the  same  group. 

Moon  (2004,  p.  164)  suggested  a list  of  assessment  criteria  for  reflection  on  the  learning  outcomes. 
According  to  the  nature  of  our  video  class,  I created  the  following  list  based  on  Moon’s  criteria  for  students’ 
reflection: 

• Appreciation  of  team  work 

• Learning  to  form  a system  for  efficient  work 

• Enhancement  of  skills  and  knowledge  /Learning  about  the  equipment  and  software 

• Project  management/Planning/W orking  long  hours 

• Motivation 

• Communicational  skills 

• Creativity 

• Preparation  ahead  of  class 

• Learning  how  organizations  work 

• Learning  about  personal  work  behavior  patterns 

• Learning  to  work  with  feedback  from  others 

• Learning  about  own  career  inspiration/Learning  to  learn  from  experience 

• Learning  self-management 

• Gaining  employability  skills/Key  skills  not  easily  gained  elsewhere  in  the  curriculum 

• Development  of  self-confidence  and  willingness  to  take  initiative 

• Suggestions  about  future  project  outcomes 

Students’  self-reflection  varied  in  length  from  half  a page  to  four  and  a half  pages.  Not  all  the  students 
addressed  every  item  on  the  list.  The  following  table  lists  the  frequency  that  the  students  who  worked  as  a group  (a 
total  of  seven  students)  addressed  for  each  item. 


Criteria 

Frequency 

Sample  Response  Coded 

Appreciation  of  team  work 

27 

We  knew  the  parameters  and  criteria  of  excellence  expected  of  this 
project.  I believe  this  was  accomplished  with  their  insight,  input,  and 
skill. 

Learning  to  form  a system 
for  efficient  work 

3 

I will  never  start  shooting  without  a perfect  production  document  again 
for  future  project. 

Enhancement  of  skills  and 
knowledge  /Learning  about 
the  equipment  and  software 

18 

I could  try  many  new  tricks  and  effects  controls  that  I have  never  used 
before. 

Project  management/ 
Planning/  Working  long 
hours 

12 

The  instructor  wanted  us  to  be  perfectly  ready  with  production 
document  before  we  start  any  shooting  or  editing;  however,  we  just 
started  shooting  with  only  simple  shooting  outline  because  we  thought 
we  could  control  the  story  with  editing  later  on.  Eventually  I could  find 
that  she  was  right  and  we  were  totally  wrong. 

Motivation 

3 

It  inspired  me  to  make  this  video  better  when  the  faculty  in  xxx  wanted 
us  to  make  a good  video  for  them,  so  they  could  actually  use  it. . . 

Communicational  skills 

7 

I hated  to  ask  around  at  first,  but  eventually  I enjoyed  meeting  with  new 
people[s]. 
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Creativity  7 

Preparation  ahead  of  class  4 

Learning  how  3 

organizations  work 
Learning  about  personal  1 

work  behavior  patterns 

Learning  to  work  with  6 

feedback  from  others 

Learning  about  own  career  3 


inspiration/Learning  to 
learn  from  experience 
Learning  self-management  3 

Gaining  employability  6 

skills/Key  skills  not  easily 
gained  elsewhere  in  the 
curriculum 

Development  of  self-  3 

confidence  and  willingness 
to  take  initiative 

Suggestions  about  future  5 

project  outcomes 


I tried  to  make  especially  the  ending  section  to  look  like  a professional 
TV  documentary  show,  with  my  own  creative  style. 

The  instructor  wanted  us  to  be  prepared  with  an  idea  for  the  final 
project  of  this  class.  I always  wanted  to  make  an  educational  video  for 
building  a computer,  so  I prepared  all  the  parts  of  the  computer  by 
disjointing  my  current  system.  (He  changed  the  topic  later) 

I thought  xxx  was  a waste  of  time  and  money  before,  but  not  any  more. 

...  I think  I was  out  of  focus  from  what  we  should  do.  With  the  advice 
of  my  partner,  however,  I could  finish  the  project  to  fit  with  the  purpose 
of  our  project. 

I showed  the  finalized  video  to  my  old  friends  who  are  currently 
working  in  a film  industry.  They  suggested  modifying  the  sound 
volumes  and  the  titles. 

The  mistakes  that  I had  in  this  project  will  not  be  repeated  in  my  future 
production  jobs,  so  I think  the  mistakes  were  worth  enough. 

This  class’  main  project  helped  me  to  learn  how  to  manage  my  time  to 
finish  my  project  in  such  a short  period. 

I found  that  the  other  classes  of  my  educational  technology  curriculum 
do  not  teach  you  as  much  about  how  to  manage  your  projects. 


Since  I had  the  experience  and  knowledge  about  Premiere  Pro,  I was 
willing  to  do  the  editing  mainly,  and  at  the  same  time,  teach  my  partner 
to  use  the  software  so  he  can  assist,  and  eventually  take  a part  of  the 
editing. 

In  the  future  project,  I would  just  make  especially  detailed  scripts  for  all 
the  actors  in  my  movie,  so  I can  shoot  all  the  scenes  in  the  way  I want. 


Frequency  was  coded  based  on  how  many  different  items  a student  mentioned  in  his/her  reflection  for  each 
criterion.  For  example,  a student  mentioned  that  he  learned  1)  improve  his  technology  skills  from  his  friends’ 
feedback,  2)  correct  his  grammatical  errors  from  his  partner’s  feedback,  and  3)  how  important  feedback  from  a third 
part  always  could  help  improve  a product,  so  his  reflection  on  “learning  to  work  with  feedback  from  others”  was 
coded  three  times.  The  numbers  in  the  table  show  us  that  students  had  very  high  appreciation  of  team  work.  They 
also  appreciated  enhancement  of  their  knowledge  and  skills.  They  appreciated  the  project  management  skills  that 
they  had  acquired  through  doing  the  project  in  a community-based  environment.  Almost  all  students  discussed  the 
above  achievements.  Not  all  students  discussed  other  criteria.  Only  one  student  mentioned  “learning  about  personal 
work  behavior  patterns.” 

The  reflection  written  by  the  two  students  of  the  group  in  the  summer  session  was  more  thorough  than  the 
reflections  written  by  the  groups  in  the  fall  semester.  The  reason  could  be  that  I provided  a written  reflection 
guideline  for  the  summer  session  students  while  to  the  fall  semester  students  I only  orally  explained  what  they 
needed  to  write  in  the  reflection.  I decided  to  provide  a written  reflection  guideline  for  the  summer  session  students 
based  on  my  experience  with  the  fall  semester  students.  I believed  that  the  fall  semester  students’  reflection  could 
have  been  more  thorough  with  a written  guideline. 

The  summer  group’s  reflection  focused  more  on  the  challenges  that  they  had  faced  and  how  they  had 
learned  from  the  challenges  while  the  fall  groups’  reflection  focused  more  on  how  much  others  had  contributed  and 
how  much  they  themselves  had  contributed  to  the  project.  The  reasons  could  be  summer  session  students  were 
challenged  more  than  the  fall  semester  students.  The  summer  session  students  had  only  five  weeks  working  on  their 
projects  and  during  summer,  schools  were  closed  and  it  was  more  difficult  for  students  to  find  community  partners. 
Even  when  students  had  a community  partner,  scheduling  was  not  easy.  The  fall  semester  students  did  not  have 
those  challenges. 

The  summer  students  who  did  the  individual  projects  debriefed  that  they  learned  technical  skills  such  as 
video  shooting,  editing,  creating  a naming  convention  for  footages  for  easier  searching,  etc.  They  at  the  same  time 
also  stated  their  frustration.  One  student  expressed  her  wish.  She  said  only  if  she  could  have  had  a teammate  who 
had  a similar  skill  level  in  understanding  computers  and  design.  She  said:  “In  this  class  I preferred  to  work  on  my 
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own  as  it  was  more  conducive  to  my  work  schedule...  I do  like  to  talk  through  my  ideas  with  other  people  though.” 
She  also  said  that  she  had  plenty  of  ideas;  however,  she  was  not  able  to  put  them  down  in  a cohesive  manner. 
Another  student  had  a very  exciting  idea  for  his  school.  He  said  that  the  school  authorities  were  also  very  excited 
with  his  anticipated  project.  However,  he  was  frustrated  when  students  did  not  show  at  his  interview  that  was 
scheduled  according  to  students’  preference.  He  said:  “It  is  summer  time!” 

Presentation  and  professional  critique 

Students  were  required  to  do  a presentation  in  the  final  week.  They  were  also  required  to  do  a professional 
critique  on  other  students’  projects.  The  groups  in  the  fall  semester  were  more  “critical.”  They  asked  questions  when 
the  other  group  was  presenting  and  offered  suggestions  about  how  to  improve  the  quality  of  the  product.  Summer 
students  were  not  that  critical.  They  almost  always  said  “great”  to  every  presentation  of  the  project.  One  of  the 
reasons  could  be  students  in  the  fall  semester  worked  in  more  depth  on  their  own  projects;  therefore,  they  had  more 
insight  about  how  to  improve  the  quality  of  a project.  Another  reason  could  be  students  in  the  summer  session  went 
through  more  challenges,  so  they  valued  more  on  every  achievement  on  the  project  that  their  peers  made  than  the 
quality  proper  of  the  project. 


Conclusion 

From  the  above  two  classes’  experience,  I would  like  to  conclude  that 

• Community-based  experiential  learning  challenges  and  benefits  students  who  do  not  have  work  experience 
more  than  students  who  have  work  experience.  According  to  students’  reflection,  those  who  did  not  have  work 
experience  addressed  more  on  their  improvement  of  communication  skills,  project  management,  and  self- 
management skills  than  those  who  were  having  a job.  They  also  addressed  more  on  gaining  own  career 
inspiration,  employability  skills,  and  key  skills  not  easily  gained  elsewhere  in  the  curriculum. 

• Community -based  experiential  learning  is  more  appropriate  to  take  place  in  regular  semesters.  It  probably 
should  not  be  scheduled  in  a summer  session.  Summer  sessions  are  usually  shorter  than  regular  semesters  and  in 
summer,  schools  and  some  other  organizations  are  closed.  There  are  not  so  many  opportunities  for  students  to 
do  a community-based  project  in  summer.  Even  if  students  get  a chance,  scheduling  of  videotaping  can  be  a 
problem.  However,  in  a more  challenging  situation,  students’  reflection  went  in  more  depth. 

• Successful  community-based  experiential  learning  requires  team  efforts.  Projects  become  much  bigger  when 
they  are  community-based  because  community-based  projects  require  social  negotiations.  Students  not  just 
learn  technology  skills  but  also  learn  communication  and  management  skills;  therefore,  they  are  involved  in  a 
multitask  environment  with  community-based  experiential  learning.  Multitask  requires  more  brain;  pulling 
heads  together  will  help  solve  problems  in  a better  way  with  less  stress. 

• Membership  of  a group  needs  attention.  New  comers  with  no  work  experience  could  have  certain  disadvantages 
in  a group  project.  If  they  are  not  aggressive  enough,  they  could  lose  some  learning  opportunities  when  the 
project  is  dominated  by  some  more  capable  students.  Concerning  this  issue,  the  instructor  needs  to  enhance 
monitoring  of  the  group  work,  for  example,  having  periodical  peer-evaluations  instead  of  having  only  one  in  the 
final  week.  In  this  way,  the  problem  that  needs  to  be  addressed  and  solved  could  be  identified  earlier  in  the 
semester. 

Designing  a community-based  experiential  learning  environment  for  students  is  feasible  for  a video 
production  class.  Moon  (2004)  said  that  “transfer  of  experiential  learning  from  an  educational  context  to  the 
situation  of  its  main  deployment  - such  as  the  work  situation  - is  a major  issue.”  (p.118).  Designing  experiential 
learning  activities  for  students  needs  to  put  an  emphasis  on  learning  in  a meaningful  environment,  in  which  students 
will  be  able  to  find  the  theme  for  the  project  that  can  serve  a real  purpose  in  the  real  world.  In  a community-based 
experiential  learning  environment,  the  shift  in  emphasis  from  faculty  teaching  to  student  learning  requires  students 
to  think  more,  participate  more,  and  take  more  control  of  their  learning  experience  (Vega  & Tayler,  2005).  “By 
working  in  groups,  students  gain  additional  insights,  are  exposed  to  multiple  points  of  view,  learn  from  each  other, 
and  learn  important  interpersonal  and  group  skills”  (Pas wan,  & Gollakota,  2004,  p225).  Peer  evaluation  and 
debriefing  encourage  students  to  draw  lessons  from  the  process  of  their  learning  experience  (Walker,  2005). 

The  limitation  of  the  above  conclusion  is  that  the  discussion  is  based  on  information  from  only  ten 
students’  learning  outcomes  in  two  semesters. 
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Rationale  behind  the  Adoption  of  Computer-Mediated  Communication  Channels 
in  an  Urban  Teachers’  Electronic  Support  System 

An  educational  technology  expert  panel  from  the  U.S.  Department  of  Education  (2000)  advocated  that 
electronic  networks,  digital  resources,  and  computer  technology  can  help  create  stronger  connections  between 
preservice  teachers,  university  faculty,  and  mentor  teachers,  and  thus  provide  valuable  resources  as  preservice 
teachers  professionally  develop  through  their  student  teaching  and  induction  phases.  Such  electronic  networks  can 
help  create  a meaningful  learning  environment  where  learning  is  touted  as  active,  constructive,  intentional, 
authentic,  and  collaborative  (Jonassen,  Howland,  Moore,  & Marra,  2003). 

The  Teacher  Electronic  Performance  Support  System  (TEPSS)  is  a vital  part  of  a PT3  project  at  a major 
southeastern  university  that  aims  to  prepare  technologically  proficient  teachers  for  diverse  urban  classrooms.  The 
design  of  the  TEPSS  system  was  based  on  the  concept  of  EPSS.  Traditionally,  EPSS  have  been  designed  and 
developed  to  improve  individual  work  performance  on-demand,  as  envisioned  by  Gery  (1991).  Currently,  the 
maturity  and  diversity  of  CMC  tools  enhanced  by  the  exponential  growth  of  computer  technology,  have  made 
leveraging  collaborative  efforts  to  promote  individual  performance  improvement  a norm.  In  accordance  with  this 
trend  and  supporting  literature  on  the  value  of  collaborative  efforts  in  teacher  education,  the  authors  have  chosen  to 
add  CMC  as  an  integral  component  of  TEPSS.  This  paper  provides  rationale  for  the  design  of  CMC  tools  to  be  used 
within  TEPSS. 

Computer-Mediated  Communication 

CMC  is  a process  by  which  different  parties,  situated  in  particular  contexts  and  separated  in  space  and/or 
time,  create,  exchange,  and  perceive  information  using  networked  telecommunications  systems  (or  non-networked 
computers)  that  facilitate  encoding,  transmitting,  and  decoding  messages  (December,  1996,  1997;  Romiszowski  & 
Mason,  2003).  CMC  can  be  synchronous  and  asynchronous.  Some  examples  of  asynchronous  CMC  tools  are  e-mail, 
listservs,  newsgroups,  bulletin  boards,  weblogs,  and  wikis.  Some  examples  of  synchronous  CMC  tools  are  chat 
rooms,  instant  messenger,  audio  teleconferencing,  and  video  teleconferencing. 

CMC  has  become  an  important  tool  in  education.  It  has  been  accredited  useful  for  fostering  learning 
through  reflection,  social  interaction,  engagement,  collaboration,  high-level  thinking,  cognitive  development,  and 
virtual  community  building  (Bloom,  1981;  Bruckman,  1998;  Carswell,  Thomas,  Petre,  Price,  & Richards,  2000; 
Fisher,  2002-2003;  Graham  & Scarborough,  1999;  Jones,  1995;  Kearsley  & Shneiderman,  1998;  Muirhead,  2001; 
Stacey,  1999).  Moreover,  a paradigm  shift  in  learning  theories  appears  to  have  taken  place  in  which  learning  is 
viewed  as  being  a process  involving  individual  learner’s  knowledge  construction  in  a collaborative  environment 
(Jonassen,  Peck,  & Wilson,  1999;  Romiszowski  & Mason,  2003).  Use  of  CMC  effectively  supports  this  idea 
through  the  high  levels  of  interaction  it  provides.  In  this  way,  it  can  enable  students  to  construct  their  knowledge 
through  active  learning  and  collaboration  (Alessi  & Trollip,  2001;  Carswell  et  al.,  2000;  Romiszowski  & Mason, 
2003).  The  following  sections  are  results  of  an  analysis  that  will  guide  the  design  of  CMC  channels  in  TEPSS. 

CMC  Channels  in  Existing  Electronic  Support  Systems 

CMC  tools  have  been  embedded  in  some  existing  electronic  support  systems  tailored  for  teacher 
professional  development.  Cal  an  dr  a , Lai,  & Sun  (2004)  reviewed  seven  such  systems, 
three  of  which  have  incorporated  CMC  into  their  models.  For  example.  Integrating 
New  Technologies  into  the  Methods  of  Education  (InTIME)  seamlessly  incorporates  a WebCT-enabled  discussion 
mechanism  into  its  system  structure.  InTIME  also  provide  the  use  of  email  for  students.  Knowledge  Loom , an 
ongoing  virtual  learning  community,  provides  several  vital  discussion  mechanisms  to  boost  knowledge  creation  and 
sharing.  Teachscape  provides  a CMC  mechanism  where  teachers  can  participate  in  virtual  discussions  either  among 
their  learning  groups  or  in  a public  arena.  Moreover,  in  Learning  Inquiry  Forum  developed  by  Indiana  University, 
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CMC  channels  such  as  email  and  chat  are  provided  to  teachers  and  students  to  share,  reflect  upon,  discuss,  and 
improve  their  teaching  and  learning  in  a smaller  group  setting. 

Theoretical  Framework 


Constructionism  and  Learning 

Constructionism  is  a theory  an  expansion  of  Piaget’s  “constructivism”  and  was  coined  by  Seymour  Papert. 
Papert  explained  constructionism  below: 

We  understand  “constructionism”  as  including,  but  going  beyond,  what  Piaget  would  call 
“constructivism”.  The  word  with  the  v expresses  the  theory  that  knowledge  is  built  by  the  learner, 
not  supplied  by  the  teacher.  The  word  with  the  n expresses  the  further  idea  that  this  happens 
especially  felicitously  when  the  learner  is  engaged  in  the  construction  of  something  external  or  at 
least  shareable....  (Papert,  1991,  p.  518) 

People  learn  best  when  they  actively  participate  in  activities  to  create  something  that  is  meaningful  to 
themselves  or  to  others  around  them  (Papert,  1993;  Resnick,  1994).  Furthermore,  such  a process  provides  a 
motivating  context  for  students  to  learn  the  subject  matter  and  content.  For  example,  when  students  are  engaged  “in 
cycles  of  designing,  evaluating,  and  redesigning,  they  also  have  the  opportunity  to  confront  their  understanding  and 
misunderstandings  of  science  concepts”  (Puntambekar  & Kolodner,  2005,  p.  185).  Constructionism  helps  to  build  “a 
context  for  learning  through  community- supported  collaborative  construction”  (Bruckman,  1998,  p.  50).  This  can  be 
done  in  a self-motivated  and  a peer- supported  manner  (Bruckman,  1998).  Pinkett  (2000)  explained  this  point  of 
view  using  the  social  cultural  constructionism  framework  that  “addresses  the  interests  of  both  an  individual  and  a 
community”  (p.  4).  According  to  him,  the  development  of  the  individuals  and  of  the  community  “are  reciprocally 
enhanced  by  independent  and  shared  constructive  activity  that  is  resonant  with  both  the  social  setting  that 
encompasses  a community  of  learners...”  (Pinkett,  2000,  p.  4).  To  scaffold  the  process  of  learning  through 
construction,  different  tools  and  agents  should  work  together  to  construct  a learning  environment  where  it  is 
essential  that  each  tool  and  agent  plays  their  unique  role  (Puntambekar  & Kolodner,  2005).  The  reason  is  that  a 
holistic  approach  to  individual  and  community  engagement  with  technology  can  better  interest  the  end-users  and 
meet  their  needs  (Pinkett,  2000). 

CMC  tools  play  a central  role  in  TEPSS.  CMC  is  not  confined  just  to  the  virtual/verbal  interactions  between 
and  among  the  learners.  It  also  expands  to  the  interactions  between  the  learners  and  the  interface,  the  resources,  and 
the  learning  environment  itself.  Each  of  the  tools  works  together  as  a whole  system  to  affect  and  effect  individual 
and  community  learning. 

Collaborative  Learning 

Even  though  research  on  cooperative  learning  has  been  claimed  one  of  the  greatest  success  stories  in  the 
history  of  educational  research  (Slavin,  1996),  collaborative  learning  presents  a more  radical  departure  from 
cooperative  learning.  Oxford  (1997)  and  Bruffee  (1999)  describe  the  difference  between  cooperative  learning  and 
collaborative  learning.  Cooperative  learning  generally  refers  to  a set  of  principles,  structures,  and  processes 
associated  with  a certain  task  that  a teacher  carefully  defines  and  monitors.  Students  in  groups  or  teams  take 
accountability  for  the  completion  of  the  task  as  a route  to  cognitive  and  social  development.  While  collaborative 
learning  enjoys  a sociocultural,  constructivist  philosophical  base,  collaborative  learning  is  more  learner  centered  and 
less  structured,  and  is  typically  applied  for  adult  learners,  especially  in  higher  education.  Collaborative  learning 
views  learning  as  construction  of  knowledge  within  a social  context  and  encourages  acculturation  of  individuals  into 
a learning  community.  Collaborative  learning  is  touted  essential  for  assisting  each  student  in  advancing  through  his 
or  her  own  zone  of  proximal  development  (Vygotsky,  1962),  where  “social  knowledge  - knowledge  acquired 
through  social  interaction  - becomes  individual  knowledge  and  individual  knowledge  grows  and  becomes  more 
complex”  (Gage  & Berliner,  1988,  p.  126).  Gardner’s  theory  of  multiple  intelligence  (1993)  also  fits  well  with 
collaborative  learning  by  emphasizing  group  work,  both  as  a function  of  distributed  intelligence  and  as  a necessary 
social  skill.  According  to  Gardner,  each  student  varies  greatly  in  his/her  strength  and  aptitude,  and  each  can  make  a 
unique  and  valuable  contribution  to  joint  task  solutions,  which  in  turn  augments  the  intelligence  level  of 
participating  individual  learners. 

Seymour  Papert  (1980)  raised  national  consciousness  about  the  potential  role  that  computer  technology  can 
play  in  creating  alternative  educational  methods  for  students.  Behind  the  current  promising  constructivist  paradigm 
shift  in  education,  technology  plays  a significant  enabler  role.  Collaborative/conversational  is  also  one  of  the 
attributes  of  meaningful  learning  envisioned  by  Jonassen,  Howland,  Moore,  and  Marra  (2003),  where  "Humans 
naturally  work  in  learning  and  knowledge-building  communities,  exploring  each  other's  skills  and  appropriating 
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each  other's  knowledge"  (p.  8).  In  their  model,  computer  technology  functions  as  a social  medium  to  support 
learning  by  collaborating  with  others,  by  discussing,  arguing,  and  building  consensus  among  members  of  a 
community,  and  by  supporting  discourse  among  knowledge-building  communities.  Starting  from  the  1990s,  the 
exponential  growth  of  computer  technology  and  the  Internet  has  made  computer- supported  collaborative  learning, 
especially  online  collaborative  learning,  a robust  alternative  to  traditional  teaching  and  learning,  and  also  made  it 
possible  for  learners  to  collaborate  even  though  they  cannot  meet  in  person  (Alavi,  1994;  Alavi  & Dufner,  2005; 
Gilbert  & Driscoll,  2002).  There  are  two  primary  modes  of  computer- supported  collaborative  learning:  synchronous 
and  asynchronous.  Computer-based  collaborative  learning  is  categorized  by  the  restriction  of  boundary  to  the 
participants  - boundaryless  participation  formats  and  boundaried  participation  formats.  Boundaryless  participation 
formats  include  the  communication  channels  which  call  for  free  participation  for  any  interested  public  learners. 
While  boundaried  participation  formats  are  the  channels  that  require  restricted  membership  such  as  WebCT  and 
listservs. 

TEPSS  presents  an  excellent  manifestation  of  how  to  take  advantage  of  computer-mediated  collaborative 
learning  to  help  professionally  and  technologically  prepare  its  urban  teachers  through  the  use  of  multiple 
synchronous  and  asynchronous  communication  channels.  These  channels  can  provide  various  levels  of  coordination 
and  cooperation  among  the  preservice  teachers  and  their  mentors.  This  can  boost  their  ability  to  reflect  on  their 
learning  and  knowledge  construction  in  cognitive,  affective,  and  behavioral  aspects  as  well  as  realte  the  reflection  to 
their  existing  knowledge  base  (Kolb,  1984;  Tennant  & Pogson,  1995).  The  practice  surely  satisfies  Pea’s  argument 
(1994)  that,  through  computer- supported  collaborative  transformative  communication,  learning  can  be  fostered 
which  facilitates  new  ways  of  thinking  and  inquiring  in  education. 

Community  of  Practice 

Lave  and  Wenger  (1991)  first  introduced  the  concept  of  a communities  of  practice  (CoP),  describing  it  as 
"...  a set  of  relations  among  persons,  activity  and  world,  over  time  and  in  relation  with  other  tangential  and 
overlapping  CoPs"  (p.  98).  The  optimal  goal  of  CoPs  is  to  promote  community  members’  knowledge-building  and 
learning  in  a specific  content  area  where  active  participation  and  reification  are  vital.  In  CoP,  newcomers  move  from 
peripheral  to  full  participation  in  the  community  as  they  learn  from  each  other.  CoP  can  promote  teachers’ 
professional  development  (Au,  2002;  Barab,  MaKinster,  Moore,  Cunningham,  & The  ILF  Design  Team,  2001; 
Schlager  & Fusco,  2004).  “Professional  development  is  viewed  as  a career-long,  context- specific,  continuous 
endeavor  that  is  guided  by  standards,  grounded  in  the  teacher’s  own  work,  focused  on  student  learning,  and  tailored 
to  the  teacher’s  stage  of  career  development.”  (Schlager  & Fusco,  2004,  p.  125)  If  a teachers’  community  is 
characterized  by  a focus  on  student  learning,  peer  collaboration,  and  reflective  dialog,  it  will  provide  social  and 
normative  supports  for  teacher  participation  in  their  professional  development  (Schlager  & Fusco,  2004). 

Learning  in  a community  can  take  a number  of  forms.  On  the  basis  of  CoP  (Lave  & Wenger,  1991; 

Wenger,  1998),  Riel  and  Polin  (2004)  further  identified  three  distinct  but  overlapping  forms  of  learning  communities 
on  the  basis  of  variation  within  their  structure  and  goals:  task-based,  practice-based,  and  knowledge-based  learning 
communities.  According  to  Riel  and  Polin,  “Practice-based  learning  communities  are  larger  groups  with  shared 
goals  that  offer  their  members  richly  contextualized  and  supported  arenas  for  learning”  (p.  20),  while  “Knowledge- 
based  learning  communities  often  share  many  of  the  features  of  a community  of  practice  but  focus  on  the  deliberate 
and  formal  production  of  external  knowledge  about  the  practice.”  (p.  21)  and  make  the  overt  commitment  to  record 
and  share  knowledge  outside  of  its  immediate  use  or  active  context. 

Practice  and  research  of  online  learning  communities  in  educational  settings  are  currently  robust  (Di  Petta, 
1998;  Schlager  & Fusco,  2004;  Swan  & Shea,  2005).  Inquiry  Learning  Forum  (1999)  is  an  online  professional 
development  system  designed  to  support  a virtual  community  of  inservice  and  preservice  mathematics  and  science 
teachers  creating,  sharing,  and  improving  inquiry-based  pedagogical  practices,  via  multiple  communication 
structures  for  members  to  engage  dialogue  with  one  another  (Barab  et  al.,  2001).  MirandaNet  (2002)  is  a web  site 
dedicated  to  providing  continuing  professional  development  for  worldwide  professionals  in  education,  and  Cuthell 
(2002)  documented  how  distributed  cognition  is  achieved  in  its  online  community.  Knowledge  Loom  (1998)  is 
another  perfect  example  of  how  online  learning  communities  can  be  created  and  maintained  around  different 
educational  topics  to  promote  teachers’  professional  development.  The  study  by  Nicholson  and  Bond  (2003)  found 
out  that,  preservice  teachers’  use  of  an  electronic  discussion  board  became  a place  for  professional  support  and 
community,  and  preservice  teachers’  reflective  thinking  developed  over  time  as  a result  of  the  discussion  board. 

The  design  and  positioning  of  CMC  channels  within  TEPSS  facilitates  two  levels  of  CoPs  - microscopic 
and  macroscopic.  Microscopically,  preservice  teachers  can  virtually  interact,  communicate,  and  collaborate  with 
their  cohorts  and  mentors  within  a specific  content  area  such  as  math,  language  and  literacy,  and  social  studies.  This 
strategy  can  help  the  cohorts  and  their  mentors  focus  on  the  topics  directly  related  to  their  future  teaching  content 
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area.  Macroscopically,  CMC  channels  also  exist  where  all  participants  and  mentors  in  TEPSS  can  freely  interact, 
communicate,  and  collaborate  with  anyone,  with  any  topics  they  are  interested  in  reflecting,  exploring,  and  sharing. 
Such  practice  can  not  only  offer  the  whole  online  community  within  TEPSS  numerous  opportunities  for  concurrent, 
purposeful,  and  sometimes,  incidental  learning,  but  also  help  build  on  community  knowledge  base  in  various  areas 
related  to  teaching. 


Stakeholder  Survey 

Eight  stakeholders  (university  faculty  from  a major  southeastern  university)  were  surveyed  in  order  to  gain 
insight  into  the  design  of  TEPSS  CMC.  As  it  was  deemed  that  faculty  would  serve  as  opinion  leaders  and  change 
agents  in  the  adoption  and  diffusion  of  TEPSS,  high  priority  was  given  to  their  needs. 

One  faculty  member  emphasized  the  importance  of  increasing  communication  among  their  preservice 
teachers  especially  while  they  were  entering  the  field-placement  stages,  “Given  the  fact  that  the  schools  that  we  used 
for  field  placements  are  usually  very  different  in  terms  of  students’  racial,  ethnic,  and  cultural  background.  The 
effective  communication  among  the  preservice  teachers  placed  in  different  schools  will  not  only  increase  their 
understanding  of  technical  aspects  of  teaching,  but  also  facilitate  their  development  of  culturally  relevant 
pedagogy.”  Moreover,  as  a result  of  the  metropolitan  location  of  the  university,  all  eight  surveyed  faculty  strongly 
recommended  the  reinforcement  of  online  communication,  as  evidenced  by  their  comments  such  as:  “Given  the 
traffic  pattern  in  the  city,  it  is  not  realistic  for  students  to  drive  back  to  campus  often  to  have  seminars  for  face-to- 
face  reflection  and  exchange”;  “Drive  time  and  the  large  geographic  space  we  have  to  deal  with  necessitate 
significant  online  communication”;  and  “Because  we  are  so  spread  out  across  the  metro  area,  we  need  a way  to 
communicate  electronically  and  efficiently.” 

Survey  results  also  indicated  that  current  CMC  channels  available  to  them  were  unsatisfactory  for  their 
teacher  preparation  courses,  “. . .was  adopted  all  year,  but  proved  to  be  very  rudimentary.  Alternative  methods  such 
as  . . . were  adopted  as  well,  however,  it  is  hard  to  getting  everyone  to  sign  up”.  Faculty  members  also  expressed  the 
need  for  extended  availability  of  communication  tools.  That  is,  rather  than  beginning  or  ending  with  a given  course 
or  program  of  study,  the  tools  would  be  available  throughout  preservice  teachers’  student  teaching  and  induction 
phases. 

The  analysis  identified  the  need  for  both  synchronous  and  asynchronous  CMC  channels  that  could  facilitate 
interactions:  a)  between  mentors  (academic  and  expert  teachers)  and  mentees  (preservice  teachers),  and  b)  among 
peers  (the  preservice  teachers).  These  results  were  in  agreement  with  a 2000  study  on  the  function  of  synchronous 
and  asynchronous  CMC  for  ESL  students.  Sotillo  (2000)  found  that  both  can  be  useful  for  different  pedagogical 
purposes.  Rourke  and  Anderson  (2002)  also  found  that  asynchronous  technologies,  such  as  email,  were  useful  for 
arranging  meetings  and  sharing  provisional  documents,  while  synchronous  technologies  were  more  useful  for 
brainstorming  and  decision-making.  Both  the  synchronous  and  the  asynchronous  CMC  users  commented  that  it 
would  be  more  productive  for  them  to  work  on  their  study  if  they  had  both  the  synchronous  and  the  asynchronous 
CMC  tools  available.  This  way  they  were  free  to  choose  an  appropriate  tool  depending  on  the  content  they  were  to 
communicate.  Bannan-Ritland  also  conducted  an  explanatory  synthesis  of  literature  related  to  CMC,  eLearning,  and 
interactivity  (2002).  Her  research  corroborates  the  role  of  both  asynchronous  and  synchronous  communication 
methods  in  promoting  learning. 


CMC  Channels  Adopted  in  TEPSS 

Based  on  a review  of  literature,  and  examination  of  some  similar  existing  systems,  and  an  initial  analysis  of 
faculty  stakeholders,  we  have  designed  a prototype  for  TEPSS  CMC.  CMC  within  TEPSS  would  be  both 
synchronous  (IM)  and  asynchronous  (email,  bulletin  board,  Blog).  Our  CMC  tools  would  be  available  to  preservice 
teacher  throughout  their  program  of  study  as  well  as  for  the  first  two  years  of  their  actual  teaching.  By  doing  so,  we 
hope  to  provide  urban  preservice  teachers  with  opportunities  to  build  online  communities,  to  construct  their 
individual  and  community  knowledge  base,  and  to  get  support  from  peers  and  experts  throughout  their  teacher 
induction  experience.  TEPSS  is  currently  undergoing  a process  of  design,  development,  and  evaluation.  As  we  now 
have  a general  idea  of  how  to  effectively  integrate  CMC  tools  within  the  system,  we  can  begin  testing  them  with  our 
end  users. 
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Introduction 

Each  of  the  authors  has  been  involved  in  team  teaching  a subject  called  Computer  Supported  Collaborative 
Work  ( CSCW)  at  Charles  Sturt  University.  The  subject  introduces  students  to  contemporary  social  and  technology 
issues  as  participants  in  online  communities.  Its  enrolment  comprises  a wide  array  of  undergraduate  and 
postgraduate  students,  studying  both  on-campus  and  via  distance  education,  throughout  Australia  and  overseas, 
hailing  from  a diverse  range  of  disciplines.  It  provides  a focus  for  discussion  and  application  of  CSCW  in  fields  such 
as  professional  development,  information  technology,  library  science,  education,  teacher  librarianship,  health  care  or 
policing.  The  four  major  outcomes  of  this  subject  are: 

1 . to  understand  the  need  for  a multidisciplinary  approach  to  learning  and  workflow  within  online 
communities; 

2.  to  work  effectively  within  a collaborative  community; 

3.  to  understand  through  negotiation  the  issues  linked  to  computer  supported  collaborative  work 
(CSCW);  and 

4.  to  demonstrate  an  understanding  of  the  processes  required  to  design,  build,  use  and  evaluate  online 
communities  using  groupware  tools. 

Students  explore  various  cognitive  frameworks  used  in  CSCW,  and  learn  how  to  select  and  tailor  a 
framework  appropriate  to  specific  collaborative  situations  and  tasks.  They  study  the  principles  underpinning  the 
design  and  building  of  workgroup  specific  infrastructures  to  support  successful  workflow  and  human  interaction.  A 
mandatory  component  of  the  subject  requires  students  to  collaborate  regularly  with  others  using  a variety  of 
software  - By  integrating  literature  and  other  subject  content  about  CSCW,  students  and  instructors  employ 
information  environments  and  groupware  tools  such  as  e-mail,  forums,  Z Object  Publishing  Environment  (Zope), 
Yahoo!  Groups,  weblogs  (blogs)  and  MOO  to  facilitate  collaborative  learning  and  knowledge  construction,  and  to 
capture  artefacts  resulting  from  these  processes. 

In  addition,  CSCW  has  a broader,  underpinning  aim  of  helping  to  nurture  community-minded  individuals, 
consistent  with  the  views  expressed  by  Peck  (1987): 

We  human  beings  have  often  been  referred  to  as  social  animals.  But  we  are  not  yet  community  creatures.  We  are  impelled  to  relate 
with  each  other  for  our  survival.  But  we  do  not  yet  relate  with  the  inclusivity,  realism,  self-awareness,  vulnerability,  commitment, 
openness,  freedom,  equality,  and  love  of  genuine  community.  It  is  clearly  no  longer  enough  to  be  simply  social  animals,  babbling 
together  at  cocktail  parties  and  brawling  with  each  other  in  business  and  over  boundaries.  It  is  our  task  - our  essential,  central,  crucial 
task  - to  transform  ourselves  from  mere  social  creatures  into  community  creatures.  It  is  the  only  way  that  human  evolution  will  be 
able  to  proceed,  (p.  165) 


Rationale 

The  subject  was  first  initiated  when  Eustace  and  Hay  (2000)  reflected  on  their  own  discourse  as  University 
teachers  and  researchers,  in  which  they  and  their  students  were  expected  to  use  a myriad  of  Internet  services  and 
tools  to  communicate  and  share  data.  They  thought  it  timely  to  develop  a subject  to  teach  both  about  and  using  such 
tools  to  help  professional  workgroups  operate  effectively  online,  based  on  a community  building  approach  or  theme. 

Since  its  genesis,  the  subject  has  evolved  at  the  hands  of  other  academics  at  CSU,  including  Mark  Lee  and 
Geoff  Fellows,  and  through  the  active  participation  and  contribution  of  a number  of  student  cohorts.  This  paper 
describes  how  the  above  objectives  were  achieved  in  the  subject’s  Spring  (July-November)  offering  in  2004,  in 
addition  to  outlining  plans  for  refinement  and  improvement  in  future  iterations. 

CSCL  and  online  learning  communities:  A brief  literature  review 

Collaborative  learning  ( CL)  evolved  from  the  work  of  Piaget  (1926)  and  Vygotsky  (1978).  It  is  based  on 
the  social  constructivist  view  that  learners  learn  best  through  positive,  cooperative  interactions  with  one  another. 
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There  is  certainly  no  shortage  of  literature  supporting  the  benefits  of  collaborative  learning  in  traditional,  face-to- 
face  settings.  Closely  related  to  this  are  the  positive  effects  that  the  social  phenomenon  of  community  can  have  on 
learning  and  knowledge  construction,  as  highlighted  by  the  work  of  Dewey  (1929),  Vygotsky  (1978),  Bruner  (1986, 
1990,  1996),  Kafai  and  Resnick  (1996)  and  Cunningham  (1996). 

Research  into  computer  supported  collaborative  learning  ( CSCL)  reveals  that  the  benefits  of  CL  can  be 
further  enhanced  through  the  employment  of  appropriate  supporting  technology  (Kaye,  1992;  Alavi,  1994;  Hiltz, 
1995;  Veerman  & Veldhuis-Diermanse,  2001).  Following  in  this  vein,  modern  information  and  communications 
technologies  can  be  put  to  use  in  the  development  of  online  learning  communities  (Bonk  & Wisher,  2000;  Hiltz, 
1998;  Palloff  & Pratt,  1999;  Rovai,  2002). 

However,  the  use  of  a suite  of  elaborate  technological  tools  or  cutting-edge  delivery  media  will  do  little 
good  to  enhance  teaching  and  learning  without  the  presence  of  well-planned  and  effective  strategies  (Clark,  1983). 
For  example,  it  is  a well-known  fact  that  active  involvement  of  the  learner  dramatically  increases  the  effectiveness 
of  the  learning.  Strategies  must  be  devised  to  ensure  each  learner  is  engaged  and  involved,  and  given  the  opportunity 
to  process  and  apply  his/her  newly  acquired  knowledge. 

It  is  also  a widely  accepted  view  that  learners  must  take  ownership  and  responsibility  for  their  learning.  As 
such,  the  role  of  the  teacher  or  lecturer  has  shifted,  in  recent  decades,  to  one  of  a guide,  or  facilitator.  In  fact,  not 
only  do  learners,  as  newcomers  to  a community  of  practice,  engage  in  “legitimate  peripheral  participation”  (Lave  & 
Wenger,  1991)  to  develop  mastery  of  knowledge  and  skills  through  interaction  with  “old-timers”  or  experts  (such  as 
their  instructors,  in  the  case  of  an  academic  environment),  they  also  have  a responsibility  - an  obligation  - to  play  a 
part  in  the  continued  evolution  and  advancement  of  the  community’s  existing  body  of  knowledge,  as  they  move 
toward  full  participation  in  the  socio-cultural  practices  of  this  community.  The  three  case  studies  of  telelearning 
innovation  presented  by  Eustace  et  al.  (2001)  and  the  Teletop  development  (see  Teletop  B.V.,  2004)  by  Collis 
(2002)  are  amongst  the  plethora  of  examples  that  stand  as  a testament  to  the  merits  of  a learning  paradigm  in  which 
instructors  focus  their  efforts  on  creating  a conducive  online  environment  for  students  to  build  content  and  take 
responsibility  for  learning. 

The  authors  have  attempted  to  work  towards  a re-usable  technology  model  and  a set  of  strategies  for 
facilitating  collaborative  learning  and  knowledge  construction  that  takes  these  factors  into  consideration,  as  well  as 
accounting  for  Salmon’s  (2004a)  five  stages  of  e-Moderating,  together  with  Gunawardena,  Lowe  & Anderson's 
(1997)  five  phases  of  social  construction  of  knowledge  in  the  online  environment  (Fig.  1). 

Groupware  framework 

The  CSCW  groupware  framework  is  centred  around  five  main  tools,  as  illustrated  in  Fig.  2. 

The  CSU  Forums  are  asynchronous,  Web-based,  threaded  discussion  boards.  The  system  used  is  one  that 
was  developed  in-house  by  the  University’s  Division  of  Information  Technology. 

Z Object  Publishing  Environment  (Zope)  (see  Zope  Corporation,  2005b)  is  an  object-oriented  web 
application  development  and  publishing  system,  written  in  the  Python  programming  language  (Fig.  3).  It  is  free  and 
open  source,  and  available  for  multiple  operating  systems.  Though  functionality  of  the  system  can  be  dramatically 
extended  through  the  use  of  Python  scripts,  a number  of  sophisticated  server- side  tasks  can  be  accomplished  with 
little  or  no  programming  knowledge,  thanks  to  Zope’s  Document  Template  Markup  Language  (DTML).  The  content 
on  a Zope  server  can  be  managed  via  a web  browser  or  through  WebDAV  (Whitehead,  2005),  the  latter  of  which 
allows  files  to  be  uploaded  directly  from  within  supporting  software.  For  example,  Microsoft  Office  documents  can 
be  saved  on  Zope  via  WebDAV,  as  if  it  were  simply  another  folder  on  the  local  network. 

Multi-User  Dungeon,  Object-Oriented  (MOO)  was  used  as  the  vehicle  for  delivering  synchronous  online 
classes  (Fig.  4).  Specifically,  the  enCore  system  developed  by  Holmevik  & Haynes  (2004)  was  used.  A MOO 
server  and  object-oriented  core  database,  is  a network-accessible,  multi-user,  programmable,  interactive  system 
originally  designed  for  the  construction  of  text-based  adventure  games,  conferencing  systems,  and  other 
collaborative  software.  Participants  (usually  called  players)  have  the  appearance  of  being  situated  in  an  artificially- 
constructed  place  (social  space)  that  also  contains  those  other  players  who  are  connected  at  the  same  time.  MOO 
facilitates  poly  synchronous  communications,  that  is  it  allows  for  a hybrid  communication  model  comprising  both 
synchronous  and  asynchronous  elements.  For  example,  players  can  interact  and  chat  in  real-time  when  they  are 
logged  in  to  the  MOO  simultaneously.  In  addition,  their  actions  can  impact  and  have  a lasting  effect  on  the  state  of 
the  objects  in  the  MOO,  even  after  they  have  logged  out  - Notes  can  be  left  on  notice  boards  and  signs  erected 
which  will  allow  messages  to  be  left  behind  for  other  players;  objects  such  as  furniture  and  office  equipment  (eg. 
whiteboards,  slide  projectors)  can  be  created,  used,  moved  and  otherwise  manipulated;  etc. 

Yahoo ! Groups  (Yahoo!  Inc.,  2005)  is  a free,  web-based  service  with  which  students  are  able  to  set  up  and 
manage  their  own  discussion  groups.  Yahoo!  Groups  works  on  a “push”  based  model  in  which  postings  to  the  group 
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are  automatically  sent  to  each  member’s  e-mail  address,  by  default.  In  addition,  a rich  set  of  ancillary  services  are 
included,  such  as  synchronous  chat  facilities,  file  and  photo  sharing  repositories,  shared  databases  and  calendars 
(Fig.  5). 

COREBlog  ( Central  Core , 2004)  is  a Zope-based,  open  source  web  logging  (blogging)  system.  Although 
originally  intended  to  allow  individuals  to  maintain  their  own  personal  journals  and  make  these  available  for  public 
viewing,  blogs  have  found  numerous  applications  in  educational  spheres.  The  easy-to-use  nature  and  informal, 
journal  entry  style  have  lent  themselves  to  ready  adoption  by  instructors,  who  create  blogs  for  purposes  ranging 
from  providing  content,  commentary  and  study  hints,  to  disseminating  subject-related  announcements.  Learners,  too, 
benefit  from  creating  their  own  blogs,  be  they  for  use  as  online  learning  portfolios  and  reflective  journals,  or  simply 
as  “soapboxes”  for  personal  self-expression.  Shared  or  group  blogs  also  exist,  which  can  serve  as  a powerful 
collaboration  and  shared  publishing  tool  (Fig.  6). 

The  abovementioned  tools  are  supplemented  with  regular  e-mail  contact  between  students  and  instructors, 
as  well  as  amongst  the  students  themselves.  Furthermore,  students  are  encouraged  to  investigate  and  explore  various 
alternative  tools  to  add  to  their  groupware  “toolkit”.  In  fact,  students  were  required  to  develop  their  own,  personal 
taxonomy  which  to  classify  and  evaluate  groupware  tools  as  they  encountered  them  throughout  the  semester. 

Teaching  and  learning  strategies 

Like  Waddoups  and  Howell  (2002),  the  authors  believe  that  the  hybridisation  of  on  and  off-campus  student 
cohorts  is  possible,  and  in  many  cases,  even  favourable.  In  CSCW,  the  diversity  is  leveraged  to  afford  students 
exposure  to  working  in  multi-disciplinary  teams,  with  members  situated  in  physically  separate  locations  and 
disparate  time  zones.  This  is,  in  many  ways,  an  accurate  reflection  of  what  is  required  of  today’s  knowledge 
workers,  who  operate  in  what  is  truly  a global  economy  dependent  on  the  Internet.  In  the  famous  words  of  Gertrude 
Stein  (1937):  “...there’s  no  there,  there.” 

The  teaching  and  learning  strategies  in  CSCW  therefore  address  the  challenges  of  creating  social  presence, 
interaction  and  a “sense  of  place”  (Coate,  1996)  in  a virtual  environment,  by  using  the  groupware  tools  discussed 
earlier  to  provide  shared  and  private  workspaces  for  learners. 

Subject  content 

In  Spring  2004,  the  subject  content  consisted  of  five  core  topics: 

1.  Underlying  principles  of  an  online  community:  The  CSCW  framework 

2.  How  to  create  online  communities:  Workgroups  and  collaborative  styles 

3.  CSCW  citizenship:  Belonging  to  an  online  learning  community 

4.  Supportive  tools  for  CSCW 

5.  Case  studies  in  CSCW 

Students  were  provided  with  an  online  schedule  of  commentary,  readings  and  exercises  for  each  topic  on 
the  subject  Zope  site  ( CSCW/online  communities  groupspace , 2004).  Exercises  in  the  topic  schedule  were  marked 
with  an  [OLR]  tag  to  notify  students  that  evidence  of  completion  of  the  task  was  to  be  published  on  Zope  in  the 
student’s  personal  folder,  which  contributed  to  his/her  own  Online  Learning  Record  (Fig.  7). 

Meetings  and  workshops 

Although  a resource-based  approach  was  adopted  in  the  presentation  of  web-based  instructional  material 
and  other  CSCW  content  to  students  via  Zope,  weekly  synchronous  online  meetings  were  held  in  Learning 
Communities  MOO  ( LC_MOO , 2004),  maintained  by  the  Internet  Special  Projects  Group  at  CSU.  To  accommodate 
both  on  and  off-campus  students,  three  one-hour  MOO  sessions  were  run  each  Thursday,  at  11:00am,  2:00pm  and 
8:00pm  Australian  Eastern  Standard  Time  (AEST).  Students  were  welcome  to  participate  in  any  or  all  of  the 
sessions.  The  daytime  sessions  were  facilitated  by  instructors  physically  present  in  the  University’s  computer  labs; 
in  the  case  of  the  evening  session,  both  students  and  instructors  attended  MOO  tutorials  via  university  or  home 
Internet  connections. 

A seminar/workshop  style  was  adopted  for  the  MOO  sessions  in  the  earlier  weeks  of  the  semester.  These 
covered  general  orientation  to  the  subject  and  its  groupware  framework,  in  particular  basic  MOO  training  and 
familiarisation  with  Zope.  Many  online  instructors  find  that  interactivity  is  preferred  over  the  one-way  information 
flow  of  lectures.  As  such,  in  later  weeks,  MOO  time  was  dedicated  to  open  discussions  and  debates  on  topics  related 
to  the  subject  content,  including  contemporary  CSCW  and  groupware  issues. 

Logs  of  all  sessions  were  saved  in  the  form  of  log  objects  in  the  MOO.  Since  the  web-based  representations 
of  MOO  objects  are  accessible  via  URLs  (Fig.  8),  hyperlinks  to  the  logs  and  lesson  slides  (contained  slide  projector 
objects)  were  able  to  be  placed  on  Zope  for  easy  access. 
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Learning  to  use  and  program  a MOO,  exploration,  R&D  and  prototype  development  of  worlds  and 
groupware,  were  also  available  using  a second,  “sandbox”  MOO  called  K9MOO  ( K9  campus  and  theme  park , 

2004).  Both  MOOs  were  available  throughout  the  session  for  students  to  hold  their  own  meetings  outside  of  regular 
class  times.  Most  students  were  able  to  build  their  own  personal  and  group  “home”  rooms,  as  well  as  populating 
these  rooms  with  their  own  MOO  objects  such  as  log  recorders,  slide  projectors  and  notice  boards,  to  enhance  the 
collaboration  environment.  Many  took  advantage  of  the  programmability  of  the  MOO  by  scripting  their  own  verbs 
(methods,  or  operations)  to  add  to  the  functionality  and  interactivity  of  their  objects. 

In  addition  to  synchronous  online  sessions,  face-to-face  meetings  were  held  for  on-campus  students.  The 
format  of  these  meetings  was  largely  informal  and  discussion-based;  they  simply  offered  opportunities  for  those 
located  at  the  University  to  convene  and  discuss/report  on  their  progress  in  the  subject.  Short  lectures  delivered  by 
the  instructors  on  alternate  weeks  were  intended  to  generate  discussion,  the  notes  for  which  were  published  on  Zope 
for  the  benefit  of  all  students. 

Forum-facilitated  discussion  and  additional  support 

The  subject  forum  was  used  by  the  instructors  to  post  announcements  on  subject-related  matters,  and  by 
students  to  obtain  general  administrative  and  technical  support.  Students  were  also  encouraged  to  participate  in  a 
continual  class  dialogue  via  the  forums,  sharing  their  reading,  experiences,  ideas  and  questions  with  their  classmates. 
For  issues  of  a more  “personal”  nature,  such  as  matters  relating  to  a student’s  own  assessment,  e-mail  was  the 
preferred  means  of  communication. 

In  addition  to  the  above,  e-mail  was  used  for  informal  interactions  between  students,  particularly  in  relation 
to  the  project  work  and  to  support  the  stages  of  group  formation. 

Pools  of  Online  Dialogue  (PODs) 

A key  component  of  the  subject  was  the  requirement  for  students  to  form  and  participate  in  small 
workgroups  called  Pools  of  Online  Dialogue  (PODs).  This  was  to  allow  them  to  explore  the  dynamics  of  the 
creation  and  maintenance  of  a such  a workgroup,  and  to  be  part  of  a supportive  group  structure  that  allowed  them  to 
explore  a deeper  understanding  of  the  subject  content/readings  and  collaborative  practice  in  general. 

Each  student  was  allocated  to  a POD  group  consisting  of  four  members.  Differing  views  exist  on  how 
groups  should  be  formed  - Some  contend  that  random  assignments  work  best  to  maximise  group  heterogeneity 
(Smith,  1985;  Fiechtner  & Davis,  1991),  while  others  favour  a more  deliberate,  manual  process  in  the  interest  of 
ensuring  fairness  of  group  composition  (Walvoord,  1986;  Connery,  1988).  Still  others  prefer  to  allow  students  the 
flexibility  of  selecting  their  own  workmates.  However,  in  CSCW,  to  leverage  the  diverse  characteristics  of  the 
individuals  in  the  class,  a deliberate  attempt  was  made  to  achieve  a mixture  of  students  studying  in  different  modes 
and  locations  in  each  POD  group.  This  was  driven  by  the  desire  for  students  to  meet  and  work  with  others  from 
backgrounds  and  interests  that  could  be  vastly  different  to  their  own.  It  also  served  to  ensure  that  the  collaborative 
work  was  in  fact  performed  online.  Many  groups  also  intentionally  consisted  of  members  studying  different  courses 
(programs)  in  various  faculties  of  the  university,  encompassing  both  undergraduates  and  postgraduates. 

The  POD  activities  began  in  Week  3 of  the  session,  when  students  used  the  subject  forum  as  an  initial 
meeting  point  to  exchange  e-mail  addresses  with  their  group  members.  One  POD  activity  was  assigned  for  each  of 
the  five  main  topics  in  the  subject.  Activities  were  posted  on  Zope  by  the  instructors  each  fortnight,  and  was  to  be 
completed  before  the  posting  of  the  next  activity. 

Although  each  POD  group  was  assigned  a number,  students  were  encouraged  to  select  a name  for  their 
POD  that  reflected  its  identity  and  purpose.  Even-numbered  PODs  were  required  to  use  Yahoo!  Groups  to  complete 
their  POD  activities,  whilst  the  odd-numbered  PODs  used  COREBlog.  (Each  POD  was  also  assigned  their  own 
shared  workspace  on  Zope,  whose  use  was  optional.)  The  purpose  of  this  was  to  afford  the  students  exposure  to,  as 
well  as  encouraging  them  to  reflect  on  the  differences  between,  contrasting  types  of  tools  and  how  they  affect 
workgroup  collaboration.  To  this  end,  the  fourth  activity  required  each  POD  group  to  send  a member  representative, 
or  “agent”,  to  participate  in  a group  using  the  other  tool.  The  fifth  and  final  activity  saw  the  agents  returning  to  their 
original,  “home”  groups  to  report  on  their  observations. 

Many  believe  that  assessment  imposes  barriers  on  effective  discussion  and  the  sharing  of  ideas  in  an  online 
learning  community  (eg.  Chen,  2004).  As  it  was  felt  that  grading  the  POD  activities  might  inhibit  students’ 
willingness  to  express  ideas  openly  and  freely,  the  decision  was  made  not  to  assess  these  activities  directly.  Instead, 
evidence  of  having  completed  the  POD  activities,  together  with  reflective  comments  on  the  experiences,  were  to  be 
incorporated  into  each  student’s  individual  Online  Learning  Record  (OLR),  which  accounted  for  a substantial 
portion  of  the  subject’s  formal  assessment. 
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In  fact,  instructors  actively  participated  in  PODs  only  where  invited  to  do  so  by  the  members,  as  “guests”. 
When  this  was  the  case,  the  guests  were  to  be  told  the  purpose  of  their  input  a given  a briefing  of  their  role.  They 
had  to  be  made  familiar  with  the  guidelines  regarding  group  processes,  provided  with  technology  support  and 
supplied  with  feedback  on  their  performance. 


Assessment  strategies 

There  were  four  assessment  items  for  this  subject.  All  four  items  were  compulsory,  available  online  and 
subjected  to  further  analysis  and  evaluation.  These  are  listed  below  in  order  of  submission: 

1.  Project  proposal 

2.  Assignment  1:  Online  Learning  Record  (OLR) 

3.  Assignment  2:  Project  report 

4.  Subject  evaluation 

The  two  major  assignments  - the  OLR  and  project  report  - were  formally  assessed,  and  each  carried  a 50% 
weighting  of  the  student’s  final  grade.  The  project  proposal  and  subject  evaluation  did  not  carry  a weighting  but 
were  required  for  successful  completion  of  the  subject. 

Students  were  advised  to  read  through  all  assessment  instructions  at  the  very  beginning  of  the  session  as 
involvement  in  online  community  building  exercises  began  in  the  first  week  of  session  and  was  ongoing  throughout 
the  semester.  They  were  also  required  to  work  out  a personal  plan  in  preparation  for  the  completion  of  weekly 
readings,  written  exercises,  practical  lab  activities,  and  collection  of  evidence  of  participation  in,  and  evaluation  of, 
online  community  activities  based  on  a supplied  framework. 

Online  Learning  Record  (OLR) 

The  Online  Learning  Record  (OLR),  after  Sy verson  (1995),  was  the  vehicle  used  to  support  knowledge 
building  and  sharing  of  concepts,  artefacts  and  experiences  throughout  the  CSCW  subject.  Students  could  also  use 
the  OLR  framework  as  a checklist  to  monitor  their  progress  in  completing  core  content.  They  were  encouraged  to 
document  or  diarise  their  journey  throughout  the  semester  by  capturing  evidence  of  all  activities  undertaken,  and 
critically  reflecting  on  their  learning.  This  may  have  included  non-mandatory  activities  undertaken  in  their  own 
time,  such  as  wide  reading  of  websites  and  journal/magazine  articles. 

Each  student  was  provided  with  a Zope  folder  or  web  space  in  which  to  create  his/her  OLR.  The  format  of 
the  OLR  was  not  stipulated  but  was  left  to  each  student’ s discretion  and  creativity  - The  process  culminated  in  a rich 
collection  of  artefacts  that  may  have  included  responses  to  the  prescribed  weekly  OLR  tasks,  blogs  containing 
reflective  comments  on  the  POD  activities,  MOO  session  logs,  project  deliverables,  annotated  bibliographies  of 
CSCW  resources,  links  to  relevant  websites  and  copies  of  e-mail  interchanges  with  instructors  and  other  students,  to 
name  but  a few. 

While  the  OLR  was  not  formally  assessed  until  the  end  of  session,  students  were  required  to  develop  the 
framework  of  their  OLR  in  Week  2 and  record  progress  on  a weekly  basis  from  Week  2 through  to  Week  12  of  the 
session.  This  required  diligence  on  the  part  of  students  to  keep  their  OLRs  up  to  date  and  not  fall  behind. 

Students’  OLRs  were  graded  against  a set  of  assessment  criteria,  based  largely  around  a weighted  five- 
point  Likert  scale.  An  excerpt  from  the  marking  sheet  used  is  shown  in  Appendix  A. 

Project  proposal  and  report 

The  project  was  applied  in  nature  and  required  students  to  work  alone,  in  pairs,  or  groups  of  three  to  report 
on,  either  the  design  and  implementation  of  a unique  online  community;  or  to  develop  a case  study  based  on  the 
practical  application  of  an  online  community  model  and/or  groupware  to  enhance  collaborative  practice  within  a 
workplace,  educational  or  entertainment  setting.  The  assignment  submission  consisted  of  a project  report  (with 
supporting  documentation,  artefacts,  additional  software,  etc.),  and  required  the  synthesis  of  ideas  and  issues  relating 
to  course  content,  as  well  as  the  analysis  and  evaluation  existing  theories,  models  and  practices  relating  specifically 
to  their  chosen  project. 

The  actual  project  topic  was  negotiated  on  a one-on-one  basis  with  a supervisor.  For  those  having  trouble 
selecting  a topic,  a list  of  additional  ideas  was  provided  on  Zope  (Appendix  B).  The  instructors  and  a number  of 
other  academics  at  the  School  of  Information  Studies,  CSU  agreed  to  act  as  “sponsors”  for  students  wishing  to 
undertake  these  projects. 

Prior  to  commencing  the  project,  students  were  required  to  complete  a Project  Proposal  form  (Appendix 
D),  which  was  reviewed  by  the  supervisor  and  appropriate  feedback  provided  via  e-mail.  The  form  was  scripted  in 
DTML  and  deployed  on  Zope.  In  many  cases  this  was  an  iterative  process,  with  students  refining  and  submitting 
several  versions  of  the  proposal  until  both  the  student/group  and  the  supervisor  were  satisfied  and  ready  to  move  on 
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will  the  actual  project  execution.  Continuous  mentoring  and  feedback  via  e-mail  continued  following  the  approval  of 
the  proposal,  throughout  the  duration  of  the  project.  The  supervisor  was  also  available  at  the  end  of  each  scheduled 
MOO  session  to  offer  additional  assistance. 

Students  were  asked  to  document  the  refinement  process  of  their  chosen  topic  and  the  subsequent 
development  of  their  project  in  their  OLRs,  but  were  reminded  that  the  OLR  itself  was  to  be  assessed  separately 
from  the  project.  Students  who  worked  in  groups  were  also  required  to  submit  a Division  of  Work  statement  so  that 
the  contribution  of  each  member  could  be  assessed.  The  assessment  criteria  from  the  marking  sheet  appears  in 
Appendix  C. 

Subject  evaluation 

The  final  assessment  item  was  the  completion  of  an  online  survey  form  evaluating  the  content,  outcomes, 
tools  and  processes  used  in  the  delivery  of  the  subject  through  a series  of  open-ended  questions.  Like  the  project 
proposal  form,  the  survey  form  was  mounted  on  Zope.  This  assessment  item  was  also  allocated  a 0%  weighting,  but 
submission  was  required  for  successful  completion  of  the  subject.  Two  copies  of  student  submissions  were 
generated  - one  stored  on  the  Zope  server  for  analysis,  and  a second  compiled  and  e-mailed  automatically  to  the 
instructors  and  respondent.  In  addition  to  eliciting  feedback  on  the  tools  and  strategies  used  in  the  subject,  the  survey 
also  served  to  prompt  students  to  reflect  summatively  on  their  experiences  over  the  semester. 

Analysis  of  student  evaluations 

Methodology 

A simple  thematic  content  analysis  approach  was  used  to  analyse  the  survey  data.  For  each  question,  all 
responses  were  first  read  at  face  value  to  produce  a preliminary  (candidate)  list  of  themes  or  issues.  This  list  was 
gradually  refined  as  subsequent  passes  were  made  through  the  data,  with  the  content  being  reviewed  in  greater  detail 
and  common  strands  factored  out.  As  part  of  this  iterative  process,  categories  were  added,  deleted,  renamed, 
combined  and  divided  as  necessary. 

Eventually,  each  response  was  categorised  according  to  the  themes/issues  identified,  to  reveal  those 
themes/issues  that  appeared  to  be  the  most  pertinent,  or  worthy  of  mention.  It  should  be  noted  that  the  categories 
were  not  mutually  exclusive;  some  responses  did  not  fall  neatly  into  a single  category,  but  rather  spanned  two  or 
more  categories.  Conversely,  other  responses  did  not  fit  into  any  of  the  categories  at  all  and  were  thus  assigned  the 
category  “OTH”  (Other). 

These  “distilled”  themes/issues  were  then  reported  on  in  the  sections  that  follow,  with  excerpts/quotes  from 
the  actual  survey  data  included  to  provide  richer  insight.  The  spelling,  grammatical  and  punctuation  errors  in  these 
excerpts/quotes  have  deliberately  not  been  rectified. 

All  in  all,  the  aim  of  the  process  was  to  attempt  to  present  a broad,  overall  or  “birds’  eye  view”  picture  of 
student  attitudes  and  reactions  towards  the  CSCW  subject,  as  seen  in  the  feedback  submitted. 

Subject  strengths 

Table  1 shows  the  categories  that  emerged  from  an  analysis  of  the  subject  strengths  listed  by  students  in 
response  to  Question  1 of  the  survey. 


Table  1.  Summary  of  responses  to  Ql:  “List  what  you  consider  to  be  the  three  strengths  of  the  subject. 
Cat.  code  Category  description  N 


(N=30) 


COM  Community-orientedness,  collaboration  and  friendliness  of  atmosphere  amongst  students  and 

between  students  and  teachers 

LEA  Learning  knowledge  and  skills  related  to  CSCW,  group  and  groupware  tools/technology 

PRO  Project 

DIV  Diversity  of  student  cohort  and  POD  workgroups 

MOO  Online  practical  sessions  (held  in  MOO) 

GRO  Opportunity  to  work  in  groups  and  develop  teamwork/collaboration  skills 

NOV  Novel/unique  experience 

FLE  Flexible,  online  nature  of  subject  (ability  to  work  from  home,  self-paced) 

LEC  Helpfulness  and  enthusiasm  of  lecturers 

DIS  Networking/interacting  with  other  students  from  the  same  discipline 

OLR  Online  Learning  Record  (OLR) 

POD  POD  activities 


13 

11 

9 


43.33 

36.67 

30.00 

26.67 
26.67 

23.33 

20.00 
20.00 
20.00 

13.33 
13.33 
13.33 
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REL 

Relevance  and  ability  to  apply  learning  to  work  situation 

3 

10.00 

TOO 

Effectiveness  of  groupware  suite/tools 

3 

10.00 

IND 

Self-directed  learning  / learning  at  an  individual  level 

3 

10.00 

CHA 

Challenging  (eg.  requiring  self-motivation) 

2 

6.67 

DED 

Inclusivity  for  distance  education  students 

2 

6.67 

EXA 

No  exam 

2 

6.67 

NON 

None  listed 

0 

0.00 

The  “COM”  category  had  the  largest  number  of  responses  associated  with  it  (13  out  of  30  students), 
indicating  that  these  students  particularly  enjoyed  the  collaborative,  community-oriented  nature  of  the  subject,  and 
the  high  levels  of  interaction  with  their  instructors  and  classmates: 

“bring  students  together  via  a different  medium” 

“Friendly  atmosphere  between  the  students  and  lecturers.” 

“Gobal  discussions  and  view  exchange” 

It  was  also  apparent  that  the  subject  content  was  well-received  by  the  students,  who  highly  valued  learning 
about  the  theory  and  practice  of  CSCW  and  groups,  while  being  exposed  to  some  of  the  many  groupware  options 
available  and  being  given  the  opportunity  to  learn  how  to  use  some  of  these  tools.  The  project  was  the  specific 
learning  activity  that  received  most  mention,  with  many  students  appreciating  the  ability  to  contextualise  their 
learning  and  apply  it  to  their  current  and/or  future  vocations: 

“The  ability  to  base  the  project  work  on  real  work  activities  - makes  it  more  meaningful  and  relevant. . .” 

“Doing  a project  that  was  directly  linked  to  something  I was  already  involved  in.” 

Another  one  of  the  issues  that  spoke  the  loudest  in  the  survey  responses  was  the  fact  that  students  highly 
valued  the  experience  of  interacting  with  others  in  the  diverse  cohort  and  workgroups: 

“Not  letting  students  chose  their  pod  groups  this  was  a great  chance  to  meet  students  in  the  same  situation  as  yourself.  Especially 
students  from  abroad.” 

“Developing  online  communities  with  students  from  diverse  backgrounds.” 

“Networking  with  studints  from  different  cultures  and  backgrounds” 

At  the  same  time,  they  benefited  from  interacting  with  those  with  similar  interests,  or  from  like  disciplines. 
A number  of  respondents  commented  on  the  novel  learning  experiences  facilitated  by  the  subject,  in 
particular  the  confluence  of  the  human  and  technological  facets  of  CSCW  and  online  communities: 

“introduction  of  unique  learning  opportunities/techniques” 

“A  bit  of  mystery  as  to  where  the  subject  was  heading  and  the  air  of  experienmentation” 

This  included  the  chance  for  them  to  work  in  teams,  and  to  develop  their  “soft”  skills  to  this  end. 

While  students  particularly  enjoyed  the  collaborative,  community-oriented  nature  of  the  subject,  its 
flexible,  online  features  were  also  applauded: 

“. . .the  subject  can  be  wholly  completed  online” 

“Learn  at  your  own  pace” 

“Flexibility  with  some  deadlines/ongoing  tasks(CSCW)  so  can  reallocate  time  where  needed.” 

Another  strength  of  the  subject  from  the  point  of  view  of  students  was  the  helpfulness  and  enthusiasm  of 
the  instructors,  which  helped  create  a supportive,  community-oriented  learning  environment.  This  was  further 
underscored  by  the  issue  of  inclusivity  for  distance  education  students. 

There  were  also  positive  comments  about  the  groupware  tools  used,  including  the  level  of  innovation  and 
the  variety  of  technologies  explored.  MOO,  especially,  was  perceived  by  many  as  a strength,  in  terms  of  its  ability  to 
provide  an  effective  yet  enjoyable  means  of  facilitating  synchronous  collaboration  and  learning. 

Hung  & Nichani  (2001)  propose  a constructivist  framework  that  suggests  e-learning  environments  should 
be  situated  in  both  the  social  community  of  practice  and  in  the  individual  minds  of  learners.  For  example,  one 
student  listed  learning  how  to  collaborate  using  groupware  tools  and  interacting  with  others  from  diverse 
backgrounds  as  major  strengths  of  the  subject,  but  also  pointed  out  that  he  benefited  from  the  personal  reflection 
afforded  by  the  OLR: 

“You  learn  to  create  an  online  learning  record,  which  in  turn  is  learning  at  an  individual  level.” 

Subject  weaknesses 

The  categories  of  subject  weaknesses  identified  are  presented  in  Table  2. 


334 


Table  2.  Summary  of  responses  to  Q2:  “List  what  you  consider  to  be  the  three  weaknesses  of  the  subject.  ” (N=30) 


Cat.  code 

Category  description 

N 

% 

TEC 

Technical  issues/difficulties 

11 

36.67 

POD 

Difficulty  in  coordinating  POD  groups  and  managing  group  dynamics/conflict 

10 

33.33 

ACT 

Fearning  activities  (eg.  number  of  practical  activities  and  case  studies,  volume  and  content  of 
readings) 

8 

26.67 

CLA 

Clarity  of  assessment  requirements  / activity  instructions 

6 

20.00 

INT 

Internal  lectures  (appropriateness,  schedule,  attendance,  content,  etc.) 

5* 

16.67 

FAC 

Fack  of  face-to-face  contact 

4 

13.33 

OTH 

Other 

4 

13.33 

TIM 

Timing/scheduling  issues  related  to  online  activities  (eg.  differences  in  time  zones) 

4 

13.33 

DIV 

Diversity  of  student  cohort  and  POD  workgroups 

3 

10.00 

MOO 

MOO  session  organisation  (chaotic,  too  much  gossip,  participants  straying  off  topic) 

3 

10.00 

ORG 

Organisation  and  structure  of  subject  content 

3 

10.00 

WOR 

Workload  / time  commitment  required 

3 

10.00 

FOR 

Fack  of  formative  assessment  and  feedback/advice  on  project  work 

2 

6.67 

NON 

None  listed 

1 

3.33 

* 16  of  the  30  respondents  were  enrolled  in  the  subject  in  internal  (on-campus)  mode 


The  most  commonly  identified  theme  in  the  responses  to  this  question  pertained  to  technical 
issues/difficulties,  such  as  login  problems  and  issues  which  arose  from  the  high  level  of  dependence  of  this  subject 
on  the  reliability  of  server  and  network  infrastructure.  The  user-friendliness  of  one  or  more  of  the  groupware  tools 
was  criticised  in  some  instances. 

The  technical  problems  were  closely  followed  by  the  difficulty  in  coordinating  and  communicating  with 
POD  group  members.  In  many  cases  this  seemed  to  be  directly  related  to  scheduling  problems,  possibly  due  to 
differences  in  time  zones.  A number  of  groups  were  faced  with  members  who  failed  to  make  adequate  contributions: 

. .after  the  first  week  we  lost  two  of  our  POD  group  members,  so  their  was  [there  were]  only  two  of  us  that  completed  the  tasks  by 
task  4, 1 think  it  was  only  me  left  in  the  group.” 

“POD  members  who  do  not  bother  to  reply  or  participate  are  a big  problem.” 

Although  diversity  was  valued  by  some  as  a subject  strength,  others  saw  the  mixture  of  students  from 
different  disciplines  within  a single  cohort  in  general,  and  within  their  POD  groups  in  particular,  as  a disadvantage: 
“Working  with  other  students  from  different  content/skills(backgrounds)  less  motivated  and  less  helpful.” 

“. . .the  cohort  was  very  diverse  and  were  starting  from  very  different  knowledge  bases  and  interest  - this  had  some  advantages  but  I 
think  more  disadvantages” 

It  could  be  argued  that  many  of  these  issues  mirror  the  demands  of  computer-mediated  communications 
and  collaborative  group  work  in  the  real  world,  which  was  one  of  the  original  intentions  of  the  subject.  In  fact,  it  was 
hoped  that  students  would  document  and  reflect  on  these  issues  in  their  OLRs,  bearing  in  mind  they  would  not  be 
directly  assessed  on  the  effectiveness  or  activity  level  of  their  POD  groups  themselves.  This  having  been  said,  more 
support  could  be  provided  to  students  in  the  way  of  strategies  for  effective  scheduling  and  organisation  of  online 
meetings.  There  may  also  be  a need  to  provide  more  motivation  and  encouragement  to  what  appears  to  be  the 
minority  of  students,  who  failed  to  actively  participate  in  the  POD  groups.  Like  O’Reilly  and  Newton  (2002),  the 
authors  believe  that  imposing  requirements  through  assessment  is  not  the  only  way  to  have  students  perceive 
importance  in  online  interaction  and  discussion. 

A significant  number  of  responses  highlighted  the  fact  that  students  sometimes  found  themselves  unsure  of 
what  exactly  was  required  of  them  in  certain  activities  and  assessment  tasks,  and  in  general.  This  is  a reminder  of  the 
importance  of  clear,  detailed  and  unambiguous  instructions  and  guidelines,  especially  in  an  online/flexible  delivery 
subject.  For  on-campus  students  this  can  be  alleviated  to  some  extent  by  providing  additional  classroom-based 
support,  although  the  ideal  level  of  face-to-face  contact  for  students  studying  the  CSCW  subject  is  unclear.  Some 
students  suggested  that  there  was  a lack  of  face-to-face  support: 

“. . .1  realise  this  is  an  online  subject  but  often  not  all  problems  can  be  answered  online.” 

On  the  other  hand,  others  felt  there  was  little  point  in  holding  face-to-face  lectures: 

“Internal  lectures  seemed  silly  for  a subject  where  practicals  and  content  were  delivered  online.” 

A number  of  students  listed  the  workload  and  time  commitment  required,  in  particular  the  large  amount  of 
reading  required,  as  a subject  weakness.  However,  it  should  be  realised  that  the  nature  of  the  subject  is  such  that  in 
order  to  be  successful,  students  must  work  consistently  throughout  the  semester.  To  use  a computing  analogy, 
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students  need  to  operate  in  “interactive  mode”  - Attempting  to  complete  the  required  tasks  just  before  the 
assignment  due  dates,  in  “batch  mode”,  is  simply  not  feasible!  One  student  admitted: 

“...the  weaknesses  I found  in  the  subject  were  more  related  to  my  lack  of  discipline  that  problems  in  the  actual  subject.” 

Difficulties  faced  by  students 

The  third  question  in  the  survey  asked  students  to  list  the  aspects  of  the  subject  they  found  most  difficult. 
The  categories  that  emerged  from  the  responses  appear  in  Table  3. 

Table  3.  Summary  of  responses  to  Q3:  “List  what  aspects  of  the  subject  you  consider  to  be  most  difficult.  (N=30) 


Cat.  code 

Category  description 

N 

% 

POD 

Coordinating  POD  groups 

11 

36.67 

SCH 

Adhering  to  the  subject  schedule 

11 

36.67 

TEC 

Resolving  technical  issues/difficulties,  including  learning/using  one  or  more  groupware  tools 

8 

26.67 

MOO 

Participating  in  and  adjusting  to  MOO  sessions  (chaotic,  too  much  gossip,  participants  straying  off 
topic) 

5 

16.67 

OLR 

Maintaining  the  OLR  and  completing  the  [OLR]  exercises 

5 

16.67 

REA 

Completing  the  prescribed  readings  (due  to  the  number,  length,  academic  language  and/or  format 
of  the  readings) 

5 

16.67 

CLA 

Understanding  the  assessment  requirements  / activity  instructions  (due  to  lack  of  clarity,  vagueness 
and/or  missing  information) 

4 

13.33 

PRO 

Completing  the  project 

4 

13.33 

OTH 

Other 

3 

10.00 

DIV 

Working  with  the  diversity  of  the  student  cohort  and  POD  workgroups 

2 

6.67 

NON 

None  listed 

1 

3.33 

Once  again,  the  resounding  issue  in  terms  of  the  aspects  of  the  subject  students  found  most  difficult,  had  to 
do  with  the  organisation  of  POD  groups.  Students  experienced  difficult  including  initiating  and  maintaining  constant 
communications  with  members,  scheduling  meetings,  encouraging  participation,  eliciting  contributions  and  reaching 
a consensus  on  topics  of  discussion.  One  student  attributed  his/her  difficulties  to: 

“Having  to  work  with  people  that  had  completely  different  goals  and  responsibilities” 

Another  student  lamented: 

“. . . Whilst  everyone  completed  their  work,  we  were  often  a member  down  when  it  came  to  discussing  responces.” 

One  student  reported  that  his  group  managed  to  overcome  the  difficulty  of  ensuring  regular  contact  by 
exercising  good  communication  skills: 

“. . .1  also  found  it  a bit  difficult  to  catch  up  with  my  group  members  regularly  due  to  the  fact  that  the  group  ha  internal  and  external 
students.  However,  good  communication  skills  that’s  shown  by  every  member  of  our  group,  solve  that  problem.” 

Concerns  in  relation  to  the  size  of  the  workload  were  also  reiterated  in  this  section,  with  many  students 
finding  it  difficult  to  work  constantly  to  stay  up  to  date  with  the  schedule  amidst  other  personal,  work  and  study 
commitments: 

“I  found  that  checking  the  forum  and  my  group  page  on  a regular  basis  was  the  most  difficult  thing  to  do  in  this  subject” 

“The  aspects. . . that  i found  most  difficult  were  trying  to  find  the  time  to  complete  every  task  on  a weekly  basis..  All  i needed  was  a 
big  assignment  and  i fell  behind  having  to  catch  up  all  the  time” 

“The  most  difficult  thing,  was  staying  in  constant  communication  with  my  POD  group,  while  trying  to  study  for  other  subjects  and 
work.” 

As  mentioned  earlier,  discipline  is  required  on  the  part  of  students  to  be  consistent  in  completing  the 
weekly  activities.  Moreover,  students  found  it  challenging  to  multitask  or  simultaneously  manage  the  various  strands 
of  activities  in  the  subject.  Amongst  the  difficulties  listed  were: 

“Juggling  the  streams  of  work  - POD,  OLR,  Project  whilst  learning  about  MOO  and  ZOPE.” 

“Unable  to  concentrate  on  a couple  of  items  moving  between  POD  activities,  CSCW  tasks,  MOOs  and  project.  Trying  to  familiarise 
oneself  with  learning  new  computer  skills  and  also  compete  tasks  that  require  reading. 

Although  the  opportunity  for  real-time  interaction  in  the  MOO  was  previously  identified  as  one  of  the 
subject’s  strengths,  one  student  described  her  experience  “mooing  with  over  20  students”  as  “chaotic  learning”.  This 
had  a lot  to  do  with  the  overwhelming  attendance  in  the  evening  session,  particularly  in  the  later  weeks  of  the 
semester,  which  a large  proportion  of  the  on-campus  cohort  began  attending  from  home  or  the  University’s  on- 
campus  residences  instead  of,  or  in  addition  to,  the  daytime  sessions. 

Suggested  improvements 

Table  4 summarises  the  responses  to  Question  4,  “List  what  improvements  could  be  made  to  the  subject.” 
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Table  4.  Summary  of  responses  to  Q4:  “List  what  improvements  could  be  made  to  the  subject.  ” (N=30) 


Cat.  code 

Category  description 

N 

% 

OTH 

Other 

7 

23.33 

MOO 

MOO  sessions  - Make  changes  to  the  number  of  scheduled  MOO  sessions,  change  the  topics 
covered  in  MOO  sessions,  better  organisation  and  more  order/control  in  MOO  sessions 

6 

20.00 

ORG 

Improve  organisation  and  structure  of  subject  content  and  resources 

6 

20.00 

POD 

Make  changes  to  POD  group  setup  and  administration  (group  size,  group  composition,  closer 
monitoring/intervention  by  instructors) 

5 

16.67 

TOO 

Changes  to  the  groupware  framework/tools 

5 

16.67 

NON 

None  listed 

4 

13.33 

OLR 

Make  changes  to  and/or  update  the  content  and/or  focus  of  the  [OLR]  exercises 

3 

10.00 

TEC 

Cater  better  for  technical  knowledge/skills  gaps 

3 

10.00 

WOR 

Reduce  the  workload  size  of  the  subject 

3 

10.00 

ASS 

Provide  more  assistance  and  feedback  with  assessment  work 

2 

6.67 

CLA 

Provide  clearer  instructions/guidelines  and  criteria  for  activities  and/or  assessments 

2 

6.67 

PRA 

Increase  the  number  of  hands-on  practicals 

2 

6.67 

REA 

Make  changes  to  the  prescribed  readings  (number,  length  and  content) 

2 

6.67 

As  can  be  seen  from  Table  4,  a large  number  of  responses  to  this  question  were  unable  to  be  classified  into 
any  of  the  identified  categories  and  were  therefore  placed  in  the  category  labelled  “OTH”  (Other).  However,  a 
noteworthy  number  of  students  made  suggestions  related  to  the  scheduled  MOO  sessions  Many  students  highly 
valued  this  component  of  the  subject,  but  expressed  the  need  for  more  order  to  these  sessions. 

In  this  and  preceding  questions,  there  were  complaints  about  the  time  and  effort  required  to  rationalise  the 
subject  content  and  assessment  requirements  and  organise  them  into  a more  manageable  construction.  This  added  an 
unnecessary  overhead,  particularly  at  the  beginning  of  the  semester.  Many  expressed  a need  to  improve  the 
organisation  and  structure  of  the  content  and  resources,  and  take  steps  to  ensure  the  consistency  completeness  and 
accuracy  of  information.  A degree  of  frustration  was  evident  in  some  students’  responses: 

“...Pertinent  pieces  of  information  were  left  off  so  that  you  spent  hours  doing  trial  and  error  to  achieve  what  could  have  been  done  in 
the  first  half  hour  if  the  instructions  were  correct. . .Old  information  on  webspace  that  was  incongruent  with  what  we  had  to  work  with 
in  a practical  session.” 

“. . .1  think  I didn’t  have  sufficient  time  at  beginning  of  course  to  extensively  read  before  realising  POD  groups  were  going  to  demand 
considerable  time  allocation.” 

A number  of  students  mentioned  specific  ways  in  which  some  of  these  concerns  could  be  addressed  to 
improve  the  subject.  Amongst  these  were  recreating  the  (Zope)  webspace  so  that  it  is  in  line  with  professional 
learning  areas,  and  developing  a more  informative  and  comprehensive  subject  outline  to  provide  a learning 
“roadmap”  and  an  overview  of  the  various  resources. 

The  difficulty  in  organising  POD  groups  arose  again,  with  students  calling  for  closer  monitoring  of  POD 
groups  and  lecturer  intervention  to  facilitate  the  initial  group  setup.  Some  students  also  stated  they  would  like  to  see 
more  technical  assistance  provided,  particularly  for  the  benefit  of  those  from  a non-information  Technology 
background.  For  example,  additional  tuition  or  simpler,  step-by-step  instructions  could  have  been  provided  for  the 
more  complex  tasks,  such  as  Zope  management  and  MOO  building/  programming.  One  student  said  he/she  would 
like  to  see  the  use  of  less  technical  language  in  the  documentation. 

Reductions  to  workload  and  volume  of  prescribed  readings  were  amongst  the  improvements  suggested: 

“OLR  topic  work  needs  to  be  reduced  whilst  the  project  is  on  - its  a big  work  load. . .1  am  still  catching  up.” 

“Need  to  rationalise  course  by  deciding  which  computer  skills/tools. . .to  develop  and  what  is  to  be  learned.” 

“. . .it  took  quite  some  effort  and  time  to  get  through  all  the  readings,  and  it  got  a little  repetative  towards  the  end  of  the  subject” 


Further  comments 

It  made  little  sense  to  quantitatively  analyse  the  responses  to  the  final  question  in  the  survey,  “Further 
comments  to  add?”  due  to  the  extremely  broad  scope  of  this  question.  Many  responses  received  here  suggested  a 
sense  of  accomplishment  and  fulfilment  by  students  in  having  completed  the  subject  and  achieving  the  intended 
learning  outcomes: 

“...it  was  satisfying  to  complete  the  major  project  and  my  olr.  Pod  and  olr  activities  provided  sound  challenges” 

“Overall  a nicely  structured  subject,  with  good  teaching  strategies,  By  studying  this  subject  i clearly  understood  the  principles  of 
CSCW,  and  how  it  can  be  applied  in  real  time  situations.” 

“. . .1  enjoyed  completing  each  OLR  and  POD  tasks.  In  the  beginning  it  took  some  time  for  contacting  each  group  members  for 
completion  of  tasks,  but  at  the  end  we  all  understood  each  other  very  well  and  contributed  our  efforts.  Thus  this  subject  indeed  teaches 
us  how  to  work  in  a group  and  also  introduces  us  with  new  ways  of  communication. 
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The  unique  learning  opportunities  and  techniques  of  the  subject  received  strong  compliments  again: 

“I  did  really  enjoy  this  subject  and  learnt  alot.  It  intorduce  me  to  a whole  new  learning  experience  through  online  collaboration.” 

. .you  don't  even  feel  like  you  are  completing  a subject. 

“I  took  on  this  subject  mainly  out  of  interest  - it  sounded  fascinating  and  it  truly  has  been.  Not  only  is  it  a new  way  of  communicating 
and  working,  but  the  subject  is  presented  like  no  other. . .1  have  thoroughly  enjoyed  my  time  here.” 

Specifically,  the  more  technically  oriented  students  benefited  from  the  socio-cultural  emphasis,  and  the 
opportunity  to  hone  their  interpersonal  and  other  non-technical  skills.  One  student  found  the  subject: 

“. . .really  enjoyable  and  completely  left  field  from  anything  else  I have  done.” 

Last  but  not  least,  the  role  of  socialisation  and  friendship  building  in  the  success  of  the  subject  was  given 
mention  in  a number  of  instances: 

“This  is  one  subject  that  really  allows  students  to  come  out  of  the  class  rooms  and  complete  the  subject  with  other  fellow  students  in  a 
more  friendly  way.” 

“I  have  learnt  a lot  from  this  subject  and  also  made  a lot  of  new  friends  which  is  very  important.  Collaboration  and  communication  is 
what  this  subject,  is  all  about,  after  all!” 


Further  work 

The  students  of  CSCW  play  an  important  role  in  the  knowledge  generation  for  the  rest  of  the  class  as  well 
as  for  and  future  cohorts.  They  therefore  have  a direct  influence  on  the  evolution  of  the  subject  and  its  content  and 
are  encouraged  to  play  an  active  role  to  this  end.  For  example,  the  artefacts  published  by  them  on  Zope  remain 
available  to  students  who  will  study  the  subject  in  the  future;  the  objects  they  have  created  in  the  MOO  persist  after 
they  have  completed  the  subject. 

The  authors  plan  to  further  refine  the  groupware  framework  by  experimenting  with  and  evaluating  other 
tools  and  technologies.  For  example,  a number  of  alternatives  exist  to  cater  for  the  subject’s  content  management 
{Content  Management  System , 2005)  needs;  even  Zope  3 (Zope  Corporation,  2005a),  is  somewhat  different  from  the 
version  used  in  the  subject.  Plone  (Plone  Foundation,  2005)  is  a powerful,  user-friendly  open  source  Content 
Management  System  based  on  Zope. 

The  authors  are  also  investigating  the  integration  of  Wiki  into  to  further  encourage  collaborative  knowledge 
generation  and  sharing,  by  allowing  students  to  annotate  and  contribute  to  the  web-based  lecture  materials  and 
online  subject  content.  Collaborative  writing  software  may  be  introduced  to  assist  groups  of  students  working  on 
their  project  reports.  Finally,  the  authors  are  exploring  the  dissemination  of  text  and  audio  content  through  the  use  of 
Really  Simple  Syndication  (RSS).  Most  blogging  systems,  as  well  as  Yahoo!  Groups,  are  capable  of  generating  RSS 
feeds  to  syndicate  XML  data  to  subscribers.  RSS  2.0  with  enclosures  allows  for  the  syndication  of  audio  content,  a 
technology  known  as  podcasting.  It  will  hoped  that  the  use  of  RSS  and  podcasting  will  make  mobile  learning  (m- 
Learning)  possible  by  catering  for  the  delivery  of  instructor  as  well  as  student-generated  content  in  the  form  of 
small,  “bite  sized”  learning  moments  viewable  on  handheld  devices  such  as  portable  music  players,  mobile  phones 
and  personal  digital  assistants  (PDAs).  For  example,  on-campus  lessons  and  face-to-face  discussions  may  be 
captured  in  MP3  format  and  podcast  for  the  benefit  of  all  students.  Students  will  be  given  the  opportunity  to  engage 
in  collaborative  activities  using  their  personal  mobile  devices. 

Furthermore,  the  authors  will  investigate  the  possible  application  of  the  online  learning  community 
building  framework  proposed  by  Brook  and  Oliver  (2003)  in  future  offerings  of  the  subject. 

Conclusion 

The  authors  believe  that  the  CSCW  groupware  framework,  as  well  as  the  teaching,  learning  and  assessment 
strategies,  can  be  replicated  or  adapted  for  most  computer  education  scenarios  that  will  benefit  from  an  online 
community  building  and  knowledge  construction  approach.  They  may  have  broader  implications  such  as 
contributing  to  best  practice  in  this  area. 

Both  the  authors’  own  observations  and  the  student  feedback  received  supply  convincing  evidence  that  the 
subject  and  its  organisation  were  well  received  by  students.  A detailed  analysis  of  forum  and  MOO  log  data  will  be 
carried  out  in  order  to  determine  the  degree  to  which  the  role  of  instructors  as  active  participants  played  an  integral 
part  in  building  group  harmony  and  confidence.  In  addition,  the  authors  plan  to  study  the  importance  and  nature  of 
mentoring  relationships  in  the  building  of  an  online  learning  community.  It  is  envisaged  that  this  will  entail 
discourse  analysis  of  e-mail,  MOO,  forum  and  POD  group  data. 

According  to  Delahoussaye  (2001,  cited  in  Differding,  n.d.)  online  education  is  “an  isolating  and  lonely 
experience”.  However,  as  one  distance  education  student  aptly  observed: 

“Studying  via  DE  can  either  be  an  isolating  experience  or  a real  online  community  connection.” 

The  framework  and  strategies  employed  in  CSCW  go  a long  way  towards  building  an  inclusive  learning 
environment  that  causes  students  - both  on-campus  and  distance  education  - to  collaborate  and  connect,  and 
encourages  them  to  evolve  from  social  animals  into  true  community  creatures. 
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Appendix  A:  Assessment  criteria  / marking  sheet  for  OLR 


1 

1 

4. 

3 

4 

5 

6 

^ l 

Readings  and  exercises 

1.  Level  of  conceptual  understanding 

2.  Use  and  evaluation  of  CSCW  tools 

3.  Object  oriented  design  and  programming 

4.  Evidence  of  effort  and  improvement  in  skills 
Collaborative  practice 

5.  Knowledge  sharing  and  action 

6.  Contribution  to  community  building 

7.  Management  of  workflow 


Presentation 

8.  Original  ideas,  comments  and  artefacts 

9.  Evidence  of  reflective  practice 

10.  Structure,  style,  logical  flow  and  referencing 

Your  final  score  out  of  50  points 
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Appendix  B:  List  of  “sponsored”  project  topics 


1.  Multimedia  interface  upgrade  for  LC_MOO  and  K9MOO 

Re-development  of  graphics  and  other  multimedia  elements  which  form  the  interface  for  LC_MOO 
(http://ispg.csu.edu.au: 8800)  as  well  as  K9MOO  (http://ispg.csu. edu.au:9000),  including  MUD  maps  for  both 
environments.  Sponsors  are  Mark  Lee  and  Ken  Eustace. 

2.  Q&A  project 

Building  a question  and  answer  Web  site  for  first  year  IT  undergraduates,  using  a collection  of  newspaper  articles  in 
XML  format.  Sponsors  are  Geoff  Fellows  and  Ken  Eustace. 

3.  Archiving  policy 

An  investigation  into  the  policy  of  archiving  data  and  back-up  procedures  over  time  in  an  organisation  eg  a school, 
business  or  government  department.  The  sponsor  is  Prof.  Ross  Harvey. 

4.  Wiki  as  a collaborative  learning  tool 

Wiki  is  a relatively  new  technology,  used  to  facilitate  collaborative  web  authoring.  The  most  well-known  Wiki 
implementation  is  Wikipedia  (http ://en. wikipedia.org) . This  project  will  involve  an  exploration  of  the  use  of  Wiki  as 
a collaborative  learning  tool  in  higher  education.  This  involves  some  technical  implementation  as  well  as  research. 
Sponsor  is  Mark  Lee. 

5.  3D  MOO  development 

Design  and  development  of  a 3D  MOO  using  ActiveWorlds  (http : //www . active  worlds . com)  to  support  collaborative 
work  in  a particular  field  such  as  business  or  education.  Sponsor  is  Mark  Lee. 

6.  Open  source  groupware  tools 

An  investigation  of  one  or  more  open  source  groupware  tools  and/or  the  development  of  a framework  using  these 
tools,  to  support  a particular  type  of  workgroup  or  community.  Sponsor  is  Mark  Lee. 
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Appendix  C:  Assessment  criteria  / marking  sheet  for  CSCW  project  report 


1 

1 

3 

4 

5 

Multidisciplinary  approach  and  problem  domain 

1.  Purpose,  plan  & timeline  are  consistent  with  outcomes 

2.  Use  & evaluation  of  CSCW  tools 

3.  Analysis  & evaluation  of  related  theories,  models  & practices 

4.  Evidence  of  application/synthesis  of  CSCW  principles  into  project 

Collaborative  practice  and  workflow 

5.  Evidence  of  individual  contribution  to  the  report 

6.  Evidence  of  individual  contribution  to  community  building 

7.  Observed  management  or  communication  with  others 

Presentation 

8.  Original  ideas,  re  sults/fm  ding  s/re  commendations  & conclusions 

9.  Quality  of  the  product,  study  or  outcomes 

10.  Structure,  style,  content,  logical  flow  & referencing 

Your  final  score  out  of  50  points 
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Appendix  D:  CSCW  project  proposal  form  items 


1.  Proposed  title  of  project: 

State  a proposed  title  for  your  project,  subject  to  change  following  consultation  with  your  lecturer. 

2.  Group  size: 

How  many  students  in  your  group?  MAX  size  = 3 

3.  Group  name: 

What  name  would  your  group  like  to  be  identified  as  (leave  blank  if  you  will  be  working  alone)? 

4.  Group  members: 

Provide  the  details  of  each  member  in  your  group.  You  are  also  required  to  nominate  ONE  member  as  the  team 
leader,  who  will  be  responsible  for  liaising  directly  with  the  lecturer. 

5.  Groupware  tool(s): 

List  the  groupware  tools  you  plan  to  use  and/or  explore  as  part  of  your  project  - e.g.  MOO,  Zope,  COREBlog, 
Yahoo!  Groups,  BSCW,  CoBrow,... 

6.  Ethics  in  my/our  research: 

Include  a brief  discussion  of  the  ethical  issues  related  to  your  research  (e.g.  privacy)  and  how  you  plan  on  addressing 
these  issues  (approx.  100  words). 

7.  Project  description: 

Include  a brief  description  of  your  project  including  client  or  sponsor,  collaborative  needs,  problems  or  concerns  to 
be  addressed  (approx.  150  words). 

8.  Expected  outcomes  of  project: 

List  the  main  outcomes  or  goals  of  your  project. 

9.  Project  plan: 

List  the  major  steps  required  to  complete  this  project  with  resources  required  (include  human  resources  here)  along 
with  a projected  timeline. 

10.  Submitted  by: 

Provide  the  name  of  the  member  submitting  this  proposal  on  behalf  of  the  group. 
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Figure  1.  Stages/phases  in  online  community  building/growth  and  knowledge  construction.  Adapted  from  Salmon 
(2004b)  and  Gunawardena,  Lowe  & Anderson  (1997) 


1 . Sharing  and  comparing  of  information 

2.  The  discovery  of  exploration  of  dissonance  or 
inconsistency  among  ideas,  concepts  or 
statements 

3.  The  negotiation  of  meaning 

4 . T esting  and  m odification  of  proposed  synthesis 
or  co  construction 

5.  Agreement  statements  and  the  application  of 
newly  constructed  meaning 


level  of 
interactivity 


Figure  2.  CSCW  groupware  framework 


Shared  web  space 
for  collaborative 
knowledge 
construction 


Instructor-led 
meetings  and 
workshops 


Pools  of  Online  Pools  of  Online  General  and 

Dialogue  (PODS)  Dialogue  (PODS)  administrative 

support 


Personal  web  space  Project  meetings 
for  Online  Learning  and  discussions 
Record  (OLR) 
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Figure  3.  The  top-level  folder  ( home  page)  of  the  CSCW  Zope  site 
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Figure  4.  The  Bulga  Ferngully  room  in  LC_MOO,  where  the  CSCW  meetings  and  workshops  were  held 
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Figure  5.  Yahoo ! Groups 


Yahoo!  Groups  : melons_r_us  - Microsoft  Internet  Explorer 
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Description  Category:  College  anti  Univeisity 

Vehicle  for  carrying  out  POD  activities  within  the  melons_r_us  group. 


Membership 
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Figure  6.  Group  blog  established  using  COREBlog 
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Figure  7.  A typical  [OLR]  entry  that  appeared  within  the  CSCW  topic  schedule  on  Zope 


File  Edit  View  Favorites  lools  Help 


* 


U flG° 


Underlying  principles  of  an  online  community: 
The  CSCW  framework 


CSCW  (Computer-Supported  Cooperative  Work)  refers  to  the  field  of  study  which  examines  the 
design,  adoption,  and  use  of  groupware.  Despite  the  name,  this  field  of  study  is  not  restricted  to 
issues  of  cooperation  or  work  but  also  examines  competition,  socialisation,  and  play  aspects  of 
groupware  use.  The  field  typically  attracts  those  interested  in  software  design,  online 
communities  and  social  and  organisational  behaviour,  including  business  people,  computer 
scientists,  organisational  psychologists,  communications  researchers,  educators  and 
anthropologists,  among  other  specialties. 


[OLR]  Exercise  1:  What  is  CSCW? 

Find  a reading  (eg.  journal  article,  book  chapter,  website)  on  CSCW,  Write  a short  paragraph 
describing  what  you  think  CSCW  is,  based  on  your  selected  reading. 

Repeat  this  task  for  the  concept  of  online  communities.  What  do  you  see  as  the  connection 
between  CSCW  and  the  development  of  online  communities? 

Post  both  the  reference  and/or  URL  for  both  your  CSCW  and  online  communities  readings,  and  the 
short  paragraph  on  your  thoughts  about  each,  to  the  CSCW  subforum  in  Week  2. 

DE  students  will  find  Reading  1 by  Preece  in  your  mail  package  useful  here.  Internal  students  can 
collect  a copy  of  this  reading  as  a handout  after  the  lecture  in  Week  1 from  the  front  office  in 
Building  05: 

Preece,  J.  2000.  Research  speaks  to  practice:  Interpersonal  communication.  In  Online 
communities:  designing  usability,  supporting  sociability.  Wiley:  Chichester,  pp.  147- 


What  is  CSCW? 


167. 
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Figure  8.  A log  recorder  object  ( behind ) player  object  (in  front)  - MOO  objects  can  be  viewed  directly  in  a browser 
by  specifying  the  relevant  object  number  in  the  URL 


3 Log  Recorder  2 (LC_MOO)  - Microsoft  Internet  Explorer 


File  Edit  View  Favorites  Tools  Help 


I » 


Address  |4J  http://ispg.csu. edu.au: 7680/ 1762/ 


Log  Recorder  2 


A simple  recording  and  playback  device.  Type  'help  $recorder'  for  operating  instructions. 
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•^1  Cl  So 


Max 


Max  relates  himself  indirectly  to  objects  around  him,  once  seen  to  be  obsessing  over  a spork  discovered  wedged 
between  the  earth  and  the  sky.  This  obsession  caused  some  to  relate  him  to  the  well-known  screen  character  "Mad 
Max",  resulting  in  rumours  of  his  dating  of  Tina  Turner.  The  lack  of  gun  toting  quickly  dispelled  the  nickname,  returning 
it  simply  to  "Max".  Though  it  is  yet  to  be  known  how  Max  managed  to  come  to  have  Tina  Turner's  number  on  speed- 
dial... 

Real  Name:  Mark  Lee 
He  is  a wizard  in  LC_MOO 

Max's  time  is  divided  fairly  equally  between  K9MOO  and  LC_MOO.  In  K9,  he  is  an  instructor  for  young  pups  new  to 
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The  Design  and  Development  of  “PEL  for  ESL”:  Exploring  the  Possibilities  of 
Problem-Based  Learning  in  the  K-12  English  as  a Second  Language 

Classroom 

Monica  Lee 
Krista  Simons 
Peg  Ertmer 
Purdue  University 

Abstract 

This  study  describes  a middle- school  English  as  a Second  Language  (ESL)  teacher's  experiences  with  a 
problem-based  learning  (PBL)  approach.  Multiple  data  were  used  to  identify  how  a PBL  approach  was  realized  in 
the  ESL  classroom  and  how  the  teacher  managed  this  unfamiliar  PBL  learning  environment.  The  collected  data 
included  video  footage  of  PBL  inquiry  activities,  classroom  observations,  and  interviews  with  the  teacher  and  the 
students.  The  study  supports  technology -enhanced  PBL  as  a beneficial  teaching  approach  for  ESL  learners  for 
authentic  language  use,  chances  to  improve  communicative  competence,  and  increased  motivation  and  self- 
confidence  as  a leaner. 


Theoretical  Framework 

Problem-based  learning  (PBL)  approaches,  originally  developed  in  medical  education,  have  been  adopted 
in  many  areas  to  provide  students  with  meaningful  learning  experiences.  Barrows  & Tamblyn  (1980)  defined  PBL 
as  “the  learning  that  results  from  the  process  of  working  toward  the  understanding  or  resolution  of  a problem”  (p.l). 
PBL  is  also  defined  as  a teaching/leaming  approach  that  relies  on  ill- structured  problem  to  provide  opportunities  for 
student  learning  (Ertmer,  Lehman,  Park,  & Grove,  2003).  According  to  Evenson  and  Hmelo  (2000),  PBL  often  has 
two  key  features;  first,  an  ill- structured  problem  used  to  prompt  students’  inquiry,  and  second,  student-centered 
learning  in  which  a teacher  takes  on  a facilitator’s  role  while  learners  take  ownership  of  their  learning.  Common 
phases  that  PBL  typically  follows  can  be  categorized  as  problem  formulation,  data  collection,  brainstorming 
solutions,  evaluating  and  selecting  solutions,  implementing  the  solutions,  and  assessment  (Seifert  & Simmons,  as 
cited  in  Scott  & Brush,  1998). 

A number  of  studies  have  examined  the  benefits  and  the  advantages  that  PBL  provides  for  language 
learning.  Abdullah  (1998)  explained  that  the  PBL  inquiry  process  places  students  in  situations  that  require  authentic 
use  of  language.  For  instance,  activities  that  engage  language  learners  in  communication  tasks,  such  as  interviewing 
others  or  being  part  of  a team,  required  them  to  communicate  with  each  other  and  deliver  the  message.  As  Krashen 
(1985)  suggested,  language  input  must  be  comprehended  in  order  to  facilitate  language  acquisition.  Students 
working  in  cooperative  groups  need  to  make  themselves  understood,  so  they  would  naturally  adjust  their  input  to 
make  it  comprehensible  (Kagan,  1985).  Forrester  and  Chau’s  article  (1999)  on  PBL  lessons  implemented  in  the 
Hong  Kong  Institute  of  Education  explained  how  the  problems  in  PBL  effectively  gave  leeway  to  students  to 
negotiate  meaning  and  to  adjust  their  language  input  to  make  themselves  understood.  In  order  to  solve  the  problems 
given,  students  had  to  collect  data  from  various  sources,  communicate  with  each  other  to  select  important  data,  and 
make  a persuasive  presentation  about  their  solutions  to  the  problem. 

Krashen  (1985)  also  suggested  that  to  facilitate  language  learning,  students  need  to  be  engaged  in 
meaningful  interaction  so  that  they  are  “concerned  not  with  the  form  of  their  utterances  but  with  the  messages  they 
are  conveying  and  understanding"  (p.  40).  McKinnon  & Rigby’s  (n.d.)  study  suggested  that  in  constructivist 
learning  environments,  language  became  the  instrument  with  which  the  students  complete  the  activities.  Hence, 
students’  attention  shifted  away  from  using  accurate  forms  of  the  language  to  meaning  of  the  message  they  were 
delivering.  Although  their  study  addressed  constructivist  learning  environments,  PBL  learning  environments  often 
include  most  of  the  fundamental  constructivist  conditions  they  referred  to,  such  as  collaborative  team  work, 
authentic  environments,  learners’  ownership  in  learning,  and  the  development  of  problem-solving  and  other 
interpersonal  skills  (McKinnon  & Rigby,  n.d.). 

Another  advantage  that  PBL  provides  for  language  learning  is  in  improving  communicative  competence. 
Kagan  (1995)  claimed  that  memorization  of  vocabulary  lists  does  not  increase  communicative  competence , the 
ability  to  use  the  language  correctly  and  appropriately  in  the  language  community  (Swain,  1985),  because  “learning 
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about  a language  is  quite  different  from  acquiring  the  language”  (p.  2).”  He  then  explained  that  the  cooperative 
group  provides  opportunities  to  produce  functional  and  communicative  language  output  that  is  critical  for  language 
acquisition.  According  to  Abdullah  (1998),  PBL  helps  language  learners  develop  their  understanding  of  the 
conventions  of  language  use  by  engaging  in  the  kinds  of  language  activity  found  in  real  life  and  not  by  learning  list 
of  rules.  McKinnon  and  Rigby’s  (n.d.)  study  with  project-based  learning  suggested  that  cooperative  group  work  and 
problem-solving  activities  would  successfully  function  as  a bridge  between  using  English  in  class  and  using  English 
in  real  life  situations. 

Inherently  increased  motivation  in  the  PBL  learning  process  may  be  another  significant  advantage  that  PBL 
offers  to  language  learners.  As  Hmelo- Silver  (2004)  explained,  enhancing  student  motivation  is  believed  to  be  a 
major  advantage  that  PBL  provides.  In  the  PBL  learning  process,  students  are  intrinsically  motivated  since  they 
work  on  a task  of  their  own  interests  and  value  what  they  are  learning  (Hmelo- Silver,  2004).  Intrinsic  motivation 
that  actively  engages  students  may  have  a strong  impact  especially  on  ESL  students  who  may  have  experienced 
more  failure  than  success  in  learning  situations  (Solomon,  2003).  ESL  students  are  often  categorized  as  at-risk 
students  in  schools  who  have  issues  of  identity,  classroom  environment,  and  effective  instruction  (Bass,  1991). 
According  to  Cerezo  (2003),  several  elements  in  the  PBL  process,  such  as  teamwork,  problem-solving  activities,  a 
focus  on  continuous  progress,  seemed  to  be  effective  strategies  to  prevent  at-risk  students  from  dropping  out  of 
school. 

In  an  attempt  to  present  the  potential  of  PBL  for  ESL  learners  and  to  provide  a guiding  model  for  ESL 
educators,  this  paper  will  describe  a middle  school  English  as  a Second  Language  (ESL)  teacher’s  experiences  with 
a PBL  approach.  The  study  will  introduce  how  a PBL  lesson  unit  was  adopted  and  applied  by  a middle  school  ESL 
teacher  in  teaching  her  ESL  students.  What  she  perceived  as  benefits  from  the  PBL  unit  will  be  also  discussed  along 
with  the  challenges  she  experienced  during  the  process.  Linally,  the  study  will  also  explain  how  her  perception 
toward  teaching  and  learning  has  changed  through  the  experiences  with  the  PBL  unit.  Accordingly,  the  research 
questions  of  this  case  study  include  the  followings: 

1.  How  was  a PBL  approach  adopted  and  applied  in  a middle-school  ESL  learning  environment? 

2.  What  benefits  and  challenges  did  a middle-school  ESL  teacher  and  students  perceive  during  the 
implementation  of  a PBL  unit? 

3.  How  has  that  teacher’s  perception  of  teaching  and  learning  in  PBL  changed  after  the  unit? 

Method 


Participants  and  Setting 

Participants  of  the  PBL  unit  were  three  7th  grade  and  four  8th  grade  ESL  students  enrolled  in  a rural  Middle- 
East  middle  school.  The  teacher  was  in  her  seventh  year  of  ESL  teaching  and  had  previously  taught  a variety  of 
grades.  All  students  were  from  Mexico  and  spoke  Spanish  as  their  first  language.  There  were  three  female  students 
and  four  male.  Most  of  the  students  were  described  by  the  teacher  as  having  a relatively  high  English  conversation 
skill  but  low  reading  and  writing  skills.  The  teacher  and  the  students  were  equipped  with  individual  lab  tops  and  a 
wireless  internet  connection. 

Research  Design  and  Role  of  Researcher 

The  research  followed  a qualitative  inquiry  framework  and  involved  description,  explanation,  and  judgment 
in  describing  an  ESL  teacher’s  experiences  with  PBL.  Multiple  data  are  used  to  identify  how  a PBL  approach  was 
realized  in  a middle  school  ESL  classroom  and  how  the  ESL  teacher  managed  this  unfamiliar  PBL  learning 
environment.  The  collected  data  included  video  footage  of  PBL  inquiry  activities,  classroom  observations  during  the 
PBL  learning  unit,  and  interviews  with  the  teacher  and  the  students. 

During  the  PBL  lesson  unit,  the  researcher  assisted  the  teacher  in  using  computer  technology  in  various 
occasions  and  also  provided  technology  workshops  to  the  students  when  needed.  The  researcher  videotaped  the 
classroom  ten  times  throughout  the  unit  and  observed  eight  times  at  various  stages,  such  as  planning,  collecting  data, 
developing,  and  making  a final  presentation.  The  observations  were  made  with  an  observation  form  developed  to 
capture  teachers’  practices  related  to  implementing  a PBL  unit.  Interviews  were  also  conducted  with  the  teacher 
three  times  (pre-,  mid-,  and  post-unit)  and  with  the  students  after  the  completion  of  the  unit. 

Data  Sources 

Video  footage.  Lor  the  purpose  of  examining  how  PBL  could  be  adopted  and  applied  in  an  ESL  learning 
situation,  the  PBL  unit  implemented  in  the  middle  school  ESL  classroom  was  videotaped  throughout  the  process. 

The  video  footage  was  utilized  in  developing  a web  site  (URL:  http://www.pblforesl.com)  whose  purpose  was  to 
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present  the  potential  of  PBL  for  ESL  learners  and  to  provide  a guiding  model  for  ESL  teachers  to  implement  a PBL 
lesson/unit  in  their  classrooms.  The  site  includes  video  footage  of  the  classroom  activities,  examples  of  student 
work,  interviews  with  the  teacher  and  the  students,  and  other  lesson  resources.  The  video  footage  served  as  the  basis 
in  examining  how  the  PBL  approach  was  realized  in  the  ESL  classroom. 

Observations.  The  classroom  was  observed  once  a week,  a total  of  eight  times,  at  the  various  stages  of  the 
PBL  lesson  unit.  The  observation  was  made  to  record  how  the  teacher  managed  the  PBL  process,  such  as 
interactions  with  students  and  classroom  management  skills  including  the  facilitation  of  students’  group  work.  The 
observations  also  aimed  to  provide  information  on  the  challenges  that  the  teacher  experienced  during  the  PBL 
process. 

Lor  classroom  observation,  a checklist  was  developed  to  capture  teachers’  PBL  practices  based  on  literature 
reviews  and  previous  observations  of  other  PBL  classrooms.  The  checklist  is  categorized  by  pedagogical 
approaches,  technology  use  for  higher-order  thinking,  planning  & organizing  techniques,  classroom  management 
skills,  collaboration,  and  professional  development  (Appendix  A). 

Interviews.  The  teacher  was  interviewed  three  times,  at  the  beginning,  in  the  middle  of  the  PBL  unit,  and 
after  the  completion  of  the  unit.  The  interviews  were  conducted  to  provide  in-depth  information  on  her  experiences 
of  the  teacher  with  a PBL  process.  The  pre-unit  interview  included  questions  on  her  expectations  and  goals  for  the 
lesson  unit  (Appendix  B),  and  the  mid-  and  post-unit  interviews  solicited  information  on  what  she  perceived  that  the 
students  gained  from  the  unit  and  what  challenges  she  experienced  (Appendix  C).  The  mid-  and  post-unit  interviews 
also  included  questions  on  how  she  dealt  with  student  guiding  and  how  her  perception  toward  PBL  has  changed 
through  these  experiences.  Interview  questions  for  students  included  what  they  liked  or  disliked  about  the  PBL  unit 
and  whether  they  perceived  the  PBL  unit  helped  them  learn  English  better  (Appendix  D). 

Results 

Research  Question  1,  How  was  a PBL  approach  adopted  and  applied  in  a middle-school  ESL  learning 
environment? 

The  PBL  lesson  unit  in  a middle  school  ESL  classroom  was  implemented  from  November  2004  to  May  2005 
with  a guiding  question  of  how  the  barriers  between  the  immigrants  and  the  community  could  be  overcome  in  a 
Middle-East  rural  area.  In  an  attempt  to  employ  a problem-based  learning  approach  in  the  classroom,  the  ESL 
teacher  sought  support  from  technology  support  staff,  technology  resources,  faculty  members  and  research  assistant 
from  a partnership  university.  She  designed  and  developed  a unit  which  was  organized  around  the  question,  “What 
do  we  want  to  share  and  teach  others  about  Hispanics  in  our  town?”.  Previous  to  the  unit,  she  carefully  planned  an 
overall  schedule  of  the  unit.  She  prepared  outside  resources  that  could  be  utilized  in  the  process  (i.e.,  a trigger  video, 
guest  speakers,  meeting  with  other  school  students,  etc.),  classroom  facilitation  methods  (i.e.,  an  individual  binder 
with  a list  of  resources,  a weekly  project  journal,  etc.),  and  assessment  tools  (i.e.,  self-assessment  rubric,  peer- 
evaluation  rubric,  final  project  rubric). 

A video  on  the  immigration  in  the  United  States  effectively  served  as  a trigger  for  the  unit.  With  the  issues 
brought  up  by  the  video,  the  teacher  introduced  problems  the  students  had  to  solve:  the  difficulties  that  non-English 
speaking  workers  encounter  in  a rural  working  place  and  how  to  resolve  the  difficulties.  Considering  students’ 
background  as  Mexican  immigrants  and  the  situations  most  of  their  parents  were  in,  the  problems  of  non-English 
speaking  workers  expected  to  bring  intrinsic  interest  and  motivation  from  the  students  and  actively  engage  them  in 
the  unit.  The  teacher  then  explained  the  overall  procedure  and  assessment  tools  of  the  unit,  such  as  a rubric  for  a 
final  product,  project  journal,  and  self-  and  peer-evaluation.  Such  assessment  tools  were  utilized  to  keep  students  on 
task  and  also  to  provide  more  opportunities  to  use  English  in  the  process. 

After  the  introduction  of  the  problem,  the  students  were  grouped  together  based  on  their  decision.  The  teacher 
allowed  the  students  to  choose  their  own  group  in  an  attempt  to  provide  more  freedom  in  the  process  and  thus  to 
motivate  them  to  actively  participate  in  the  activities.  Three  students  were  grouped  to  take  charge  of  collecting  data, 
while  two  other  students  were  given  responsibility  in  writing  articles  and  publishing  them.  Two  remaining  students 
were  also  given  a role  of  assistants.  Each  different  role  was  assigned  to  facilitate  the  group  activities  and  discussion. 
However,  after  group  activities,  students  were  gathered  to  share  ideas  and  make  an  important  decision  together. 

Students  began  to  investigate  the  issues  of  immigrants  through  the  Internet  search  provided  by  the  teacher.  After 
spending  some  time  to  collect  data,  they  were  gathered  together  to  discuss  the  data  they  collected  and  make  a 
decision  on  what  they  would  like  to  do  about  the  problems.  After  discussing  the  problems  and  data  collected,  the 
students  decided  to  develop  a movie  that  would  describe  the  difficulty  of  non-English  speaking  workers.  The  teacher 
and  the  students  determined  that  a movie  on  the  difficulty  of  non-English  immigrant  workers  would  help  the 
community  gain  an  understanding  of  new  immigrants  and  thus  increase  common  experiences  and  understanding 
needed  to  overcome  a barrier  between  the  immigrants  and  the  community. 
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To  provide  real-world  perspectives  on  the  issues,  an  employee  who  had  experiences  of  working  with 
immigrant  workers  was  invited  as  a guest  speaker.  Before  the  guest  speaker  session,  the  students  were  given  time  to 
develop  questions  for  the  speaker  and  practiced  the  questions.  With  the  teacher  facilitating  the  discussion,  students 
asked  their  questions  to  the  quest  speaker.  In  addition,  a professor  who  taught  Spanish  in  a community  college  was 
also  invited  to  provide  multiple  perspectives  on  the  problems.  Following  the  same  procedure,  the  students  developed 
questions,  practiced  their  questions,  and  asked  their  questions  to  the  guest  speaker. 

Based  on  the  data  collected  from  various  sources,  such  as  the  Internet,  guest  speakers,  and  discussions,  the 
students  began  to  write  a movie  script.  Writing  a script  took  much  time  since  students  all  needed  to  agree  on  main 
characters  and  the  storyline.  Having  an  individual  lap-top  was  beneficial  in  this  stage  since  they  all  could  develop 
ideas  whenever  they  wanted  and  shared  them  in  the  classroom.  With  the  script  written  by  and  agreed  by  all  of  the 
students,  they  began  filming  a movie,  using  a digital  camera,  with  support  and  help  from  the  researcher  and  students 
of  a local  high  school  film  class.  As  a final  stage,  they  presented  their  movie  with  the  community  members, 
including  teachers  and  students  of  the  school,  local  business  employees,  and  their  family  members. 

In  the  meantime,  in  order  to  have  more  opportunities  to  read  and  write  in  English,  the  students  published  a 
newspaper  three  times,  including  articles  they  wrote.  The  articles  discussed  the  problems  of  immigrants  and  their 
opinions  regarding  such  problems.  A number  of  English  native  students  and  teachers  along  with  other  ESL  students 
outside  the  classroom  also  participated  in  writing  their  opinions  on  the  issues.  Writing  articles  and  publishing  a 
newspaper  were  expected  to  provide  opportunities  in  improving  not  only  English  reading  and  writing  skills  but  also 
communication  skills  to  share  their  ideas  with  others. 

Seifert  & Simmons  (1997)  explained  that  common  phases  that  PBL  typically  follows  are  problem 
formulation,  data  collection,  brainstorming  solutions,  evaluating  and  selecting  solutions,  implementing  the  solutions, 
and  assessment  (as  cited  in  Scott  & Brush,  1998).  Although  the  PBL  unit  implemented  in  the  ESL  classroom  did 
not  exactly  follow  the  six  general  stages  of  PBL,  the  overall  procedure  of  the  PBL  unit  can  be  explained  following 
the  six  stages: 

1 . Problem  formation 

1)  Video  on  ‘immigration’  of  the  United  States 

2)  Introduction  to  the  problem:  issues  and  problems  that  non-English  speaking  workers  face  in  a 
rural  Indiana  and  ways  to  overcome  the  barrier  between  the  community  and  the  workers. 

2.  Data  collection 

1)  Information  search  through  the  Internet  on  immigrants  in  Indiana  and  in  the  community 

2)  Discussion  with  a guest  speaker,  an  employee  who  had  experiences  of  working  with  non- 
English  speaking  workers,  and  the  community  members,  such  as  a non-English  speaking 
worker  and  a Spanish  professor 

3.  Brainstorming  solutions 

1)  Writing  a project  journal  along  with  self-  and  peer  -evaluation 

2)  Writing  articles  about  issues  related  to  non-English  speaking  immigrants  in  the  community 
and  also  in  the  school 

3)  Discussion  in  group  and  as  a whole  class 

4.  Evaluating  and  selecting  solutions 

1)  Publishing  newspapers  for  other  students  in  the  school  to  read  the  articles  they  wrote  about 
the  issues 

2)  Discussion  in  group  and  as  a whole  class 

5.  Implementing  the  solution 

1)  Writing  a movie  script  that  describes  complex  issues  facing  non-English  speakers  working  in 
rural  area 

2)  Filming  and  editing 

3)  Developing  an  informational  package  to  help  local  companies  and  agencies  better  understand 
non-English  speaking  workers 

4)  Presenting  the  movie  to  community  members 

6.  Assessment 

1)  Final  project  rubric  and  self-  and  peer-  evaluation 

Observation  checklist  revealed  that  she  carefully  planned  and  organized  learning  resources  and  activities 
prior  to  the  classroom,  with  work  sheets  or  questions  to  answer.  In  addition,  she  often  integrated  technology 
components  in  the  process.  In  the  stage  of  brainstorming,  she  asked  the  researcher  to  utilize  Inspiration  program  to 
help  students  represent  concepts  and  relationships  between  their  ideas.  Students’  progress  was  monitored  frequently 
through  a project  journal  required  every  other  week.  She  also  successfully  collaborated  with  other  teachers 
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throughout  the  process,  asking  to  write  articles  for  a newspaper  and  sending  a newspaper  to  share  with  their 
students. 

Research  Question  2,  What  benefits  and  challenges  did  a middle-school  teacher  and  students  perceive  during  the 
implementation  of  a PBL  unit  ? 

The  pre-unit  interview  indicated  that  her  goal  to  accomplish  through  the  PBL  unit  was  to  provide  the 
students  with  as  many  opportunities  as  possible  to  read,  write,  speak  and  listen  in  English.  In  the  post-unit  interview, 
she  stated  that  she  believed  that  her  goals  for  the  unit  have  met,  considering  a number  of  opportunities  the  students 
had  in  the  PBL  unit  in  reading,  writing,  speaking,  and  communicating  in  English.  Throughout  the  unit,  the  ESL 
students  wrote  a project  journal  once  a week  and  communicated  with  outside  the  classroom  speakers  with  the 
questions  they  developed.  While  writing  articles  for  the  newspaper,  they  also  communicated  with  other  students  and 
teachers  asking  for  their  opinions.  Writing  a movie  script  and  filming  it  with  a high-school  film  crew  also  engaged 
students  in  a number  of  activities  in  using  English. 

In  the  post-unit  interview,  she  expressed  her  satisfaction  and  excitement  on  her  PBL  unit,  stating  that  “Kids 
are  speaking  and  communicating  more  than  I’ve  ever  imagined.”  She  also  expressed  her  strong  satisfaction  with  a 
PBL  teaching  and  learning  approach  and  stated  that  she  believed  students  have  gained  a lot  of  things  from  the  PBL 
experiences: 

“They  are  still  learning  speaking,  speaking,  listening,  reading,  and  writing.  But  to  them,  it  doesn’t 
seem  like  learning,  but  they  are.  Instead  of  reading  from  the  text  and  answering  questions,  they  are 
talking  about  things  that  concern  them  in  their  community  and  in  their  life.  And  they  are  working 
for  solutions  or  suggestions  to  solve  those.  In  doing  that,  they  are  using  English,  reading,  writing, 
comprehending. . . and  they  are  saying  back.  All  the  facets  of  language,  they  are  using  them.” 

In  the  post-unit  interview,  she  also  remarked  that  the  positive  feedback  that  she  and  her  students  have  been 
getting  from  the  school  and  from  the  community  was  one  of  benefits  that  students  gained  from  the  unit.  A number  of 
teachers  in  the  school  used  the  newspapers  published  by  the  ESL  students  as  a reading  material  in  their  classroom. 
Such  activities  naturally  generated  opportunities  for  the  ESL  students  to  discuss  the  issues  with  other  teachers  and 
other  students  in  the  school.  She  believed  that  her  PBL  unit  positively  affected  on  the  motivation  and  self- 
confidence  of  her  students,  who  were  often  silenced  in  the  mainstream  classroom. 

The  mid-  and  post-unit  interviews  indicated  some  of  the  challenges  she  encountered  during  the 
implementation  of  the  PBL  unit.  She  stated  that  the  most  difficult  thing  she  experienced  was  a time  management  to 
finish  learning  activities  on  time  and  maintain  the  time  schedule  with  other  requirements  from  the  school.  Along 
with  the  PBL  unit,  the  students  were  expected  to  receive  help  and  support  to  successfully  follow  mainstream  classes 
from  the  ESL  teacher.  Therefore,  in  several  occasions,  the  teacher  had  to  re-schedule  her  plan  for  the  PBL  unit  in 
order  to  deal  with  other  problems  that  the  students  brought  from  other  classes.  In  addition,  she  explained  that  often 
students’  activities  took  more  time  than  she  expected,  because  of  their  low  level  English  reading  and  writing  skills. 
She  also  had  to  deal  with  regular  school  issues  like  students’  absence  and  administrative  works  related  to  the 
management  of  ESL  program  in  the  school. 

To  solicit  explicit  information  on  the  challenges  she  encountered,  the  researcher  provided  a number  of 
challenges  to  the  teacher  and  asked  her  to  rank  them  accordingly.  The  challenges  included  a lack  of  knowledge  and 
skills,  feedback  and  expectation,  reward  and  incentive,  motivation,  resources,  and  school  culture.  She  ranked  a lack 
of  systematic  support  and  preparation  time  as  a first  barrier  and  a lack  of  feedback  and  expectation  as  a second 
barrier.  She  selected  a lack  of  motivation  as  a third  barrier  because  of  the  burden  of  overwhelming  works  that 
accompanied  the  PBL  unit.  She  explained  that  although  it  was  exciting  and  motivating  to  see  her  students  doing 
what  concerned  them  in  their  life  and  voicing  their  opinions  in  the  school,  at  the  same  time,  the  workload  was  very 
burdensome  for  her.  A lack  of  reward  and  incentive  and  a lack  of  knowledge  and  skills  followed  respectively. 

When  asked  difficulties  she  had  at  different  stages  of  PBL,  she  responded  that  developing  a movie  was  the 
most  difficult  part  because  of  scheduling  the  students  and  other  support  groups’  time.  Filming  the  movie  brought  up 
other  issues  like  having  the  students  stay  after  school,  arranging  transportation  for  the  students,  getting  permission 
to  go  to  a different  place,  etc.  Some  of  the  difficulties  she  experienced  in  brainstorming  and  collecting  data  included 
preparing  different  examples  at  various  English  levels  in  order  to  help  the  students  at  various  English  levels 
understand  the  message.  However,  she  stated  that  she  didn’t  feel  it  difficult  to  facilitate  students’  group  work 
because  they  chose  what  group  they  wanted  to  be  in  and  they  all  worked  well  together.  She  believed  a self-decision 
of  a group  contributed  to  their  involvement  and  engagement  in  the  group  work.  She  explained  the  easy  of  facilitating 
students’  group  work  also  came  from  knowing  the  students  well.  For  example,  she  knew  that  one  student  often  had  a 
strong  opinion,  so  she  assigned  the  student  to  write  articles  by  herself  while  accomplishing  a collective  goal  for  her 
group  to  publish  a newspaper. 
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All  the  students  participated  in  the  PBL  unit  responded  very  positively  regarding  their  experiences  with  the 
PBL.  All  answered  that  they  enjoyed  the  process,  some  explaining  their  satisfaction  due  to  group  work  and  use  of 
technology  and  others  due  to  things  they  did  outside  the  textbooks.  However,  some  of  them  also  expressed  their 
dissatisfaction  with  heavy  workload  and  longer  time  of  the  PBL  unit,  saying  “It’s  taking  too  much  time,  and  I’d  like 
to  move  onto  a different  topic.”  For  the  question  on  whether  it  helped  their  English  skills,  most  of  them  agreed  that  it 
helped  them  practice  English,  two  of  them  weren’t  sure  about  whether  the  PBL  process  explicitly  helped  them  in 
learning  English. 

Research  Question  3,  How  has  that  teacher's  perception  of  teaching  and  learning  in  PBL  changed  after  the  unit? 

Video  footage  and  observations  in  the  classroom  revealed  that  she  had  a strong  belief  in  student-centered 
learning.  Students  were  often  given  responsibility  in  learning  activities,  such  as  selecting  their  own  groups,  deciding 
on  what  to  do  with  the  problem.  For  instance,  the  students  interacted  with  guest  speakers  mostly  by  themselves, 
asking  questions  and  responding  to  the  speaker’s  question.  In  addition,  the  teacher  often  tried  to  motivate  and 
empower  the  students  by  reminding  them  of  the  importance  of  what  they  were  doing. 

Regarding  her  perception  change  toward  teaching  and  learning  through  PBL,  she  responded  very  positively 
about  the  impact  of  her  PBL  in  the  post-unit  interview. 

“I  experienced  the  power  [of  PBL].  A lot  of  teachers  don’t  realize  it.  You  have  to  see  what  kids 
could  learn.  Many  of  my  students  are  coming  back  with  a whole  page  of  descriptive  writing.  They 
were  barely  writing  a couple  of  sentences  when  this  unit  began.  It  [PBL]  is  just  a wonderful 
teaching  and  learning  tool,  getting  them  out  of  the  book.” 

Discussion 

Data  collected  through  video  footage,  observations,  and  interviews  revealed  how  a problem-based  learning 
approach  was  adopted  and  applied  in  a middle-school  ESL  learning  environment.  A middle-school  ESL  teacher 
perceived  that  a PBL  unit  was  successfully  adopted  and  applied  in  ESL  learning  environments,  actively  engaging 
students  in  the  learning  activities  and  positively  affecting  their  motivation  and  self-confidence.  Video  footage  and 
observation  data  also  indicated  that  a successful  implementation  of  a PBL  may  require  a number  of  instructional 
strategies,  such  as  careful  planning  and  preparation  of  learning  resources  and  activities,  use  of  technology,  and 
frequent  monitoring  of  students  progress. 

Interestingly,  the  teacher  did  not  perceive  students’  low  level  of  English  skills  as  the  serious  challenges  for 
PBL.  Suggestions  made  by  the  teacher  may  provide  important  insight  of  what  is  needed  to  implement  PBL  with  ESL 
students.  She  recommended  balancing  the  information  into  the  level  of  English  that  kids  are  at:  “If  they  are  beginner 
English  level,  it  will  have  to  be  really  scaled  down  to  their  language  ability.”  She  also  suggested  preparing  a lot  of 
practical  examples  of  various  levels  of  English  so  that  students  at  different  level  could  all  understand  what  they  were 
expected  of. 

However,  some  difficulties  were  also  experienced  by  the  teacher  in  the  process.  The  most  difficult 
challenge  the  teacher  experienced  was  a time  management  between  PBL  activities  and  other  school  requirements, 
such  as  administrative  works,  homework  from  other  classrooms.  In  addition,  in  spite  of  motivating  factors  of  PBL, 
the  teacher  explained  that  the  workload  to  prepare  PBL  learning  activities  was  heavy.  As  Perkins  (1991)  points  out, 
the  cognitive  demands  that  place  on  teacher  and  learners  tend  to  increase  in  [constructivist]  instruction  (as  cited  in 
Driscoll,  2000,  p.  381).  Unfortunately,  these  increased  cognitive  demands  may  prevent  a teacher  from  applying  a 
constructive  instruction,  such  as  PBL.  As  Krajcik,  Blumenfeld,  Marx,  and  Soloway  (1994)  suggested,  if  any 
innovation,  such  as  PBL  approaches,  is  to  be  adopted  successfully,  teachers  may  need  additional  help  to  overcome 
the  difficulties,  such  as  heavy  workload  or  longer  preparation  time. 

The  researcher  hopes  that  the  through  this  paper  that  described  an  ESL  teacher’s  experiences  with  PBL, 

ESL  administrators  and  educators  may  gain  some  insight  into  how  this  problem-based  learning  can  be  adopted  and 
applied  in  ESL  learning  situations.  More  importantly,  it  is  also  hoped  that  through  this  paper,  ESL  educators  would 
find  it  motivating  and  exciting  to  attempt  problem-based  learning  approaches  in  their  classroom.  Further  dialogue  on 
implementing  this  student-centered,  inquiry-based  problem-based  learning  approach  in  ESL  learning  environments 
would  benefit  many  ESL  educators,  providing  promising  ways  to  effectively  teach  ESL  learners. 
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Abstract 

With  the  combination  of  innovative  technological  developments  and  evolving  notions  about  learning,  the 
interactive  and  collaborative  learning  is  gradually  emphasized  in  online  learning  environments.  However,  current 
evaluation  approaches  are  not  appropriate  to  evaluate  so cio- cultural  aspects  of  online  learning  environments.  This 
paper  details  a theoretical  framework  for  evaluating  online  learning  environments,  especially  interaction  among 
learners,  tools,  and  outcomes,  based  on  activity  theory.  Implications  of  the  framework  for  future  research  are  also 
presented. 

Learning  is  considered  as  an  active  process  for  constructing  knowledge.  Guiding  and  supporting  learners 
for  their  construction  of  their  understanding  has  been  emphasized  in  learning  environments  as  well  as  instruction 
(Wilson,  1996;  Bonk  & Cunningham,  1998).  Since  the  early  1990s,  designing  meaningful  learning  environments  has 
been  one  of  the  most  attractive  issues  to  many  of  instructional  designers  in  the  field  of  instructional  technology. 
Some  of  the  examples  are  Web-Based  learning  environments,  constructivist  learning  environments,  student-centered 
learning  environments,  open-ended  learning  environments,  technology-mediated  learning  environments,  and  online 
learning  environments. 

A learning  environment  is  a place  where  learners  can  access  resources  and  construct  meaningful  knowledge 
with  teachers’  guidance;  thus,  learning  is  intentionally  fostered  and  supported  in  a learning  environment  (Khan, 
1997).  The  development  of  Web-based  technologies  has  enabled  people  to  create  effective  learning  environments 
via  the  Web.  Thus,  online  learning  environments  use  the  Web  as  a supplement  to  face-to-face  instruction,  a mixed 
mode  with  face-to-face  instruction,  or  exclusively  Web-based  instruction  without  face-to-face  components  (Mishra, 
2002).  Online  learning  environments  can  provide  learners  with  such  advantages  as  flexibility  in  time  and  place  and 
easy  access  to  learning  materials.  Online  learning  environments  in  many  cases  are  based  on  constructivist,  learner- 
centered  principles,  and  learners  are  expected  to  share  their  perspectives  and  to  collaboratively  construct  the 
knowledge  within  a group  of  learners  (Gunawardena,  Carabajal,  & Lowe,  2001).  Because  interactive  and 
collaborative  learning  has  proved  to  be  effective  to  generate  meaningful  learning  activities  (Jonassen  & Rohrer- 
Murphy,  1999),  they  have  been  gradually  gaining  attention  in  online  learning  environments. 

Evaluating  online  learning  environments  has  become  more  important  as  interests  and  applications  of  online 
learning  increase.  Just  as  “the  situated  character  of  learning  was  complex  and  students  had  to  maneuver  within  a 
complex  system”  (Havnes,  2004,  p.174),  evaluation  of  students’  learning  outcomes  and  instruction  should  analyze 
the  complex  system  or  the  learning  environment.  Learning  cannot  be  properly  assessed  solely  on  the  analysis  of 
individual  students’  interpretation  of  instruction;  rather,  evaluation  approaches  should  include  how  learners 
construct  knowledge,  how  and  what  teachers  teach,  and  how  other  factors  influence  their  learning  in  online  learning 
environments  (Havens,  2004).  A variety  of  evaluation  models  and  approaches  have  been  developed  to  examine 
interaction,  usability,  cognitive  processes,  different  ways  of  learning  assistance,  and  educational  software.  However, 
evaluation  approaches  used  to  evaluate  the  effectiveness  of  learning  in  classrooms  are  insufficient  to  evaluate 
dynamic  and  complex  online  learning  environments  (Gunawardena  et  al.,  2001).  Thus,  a genuine  need  has  emerged 
to  develop  a new  evaluation  approach  appropriate  for  online  learning  environments. 

The  purpose  of  this  paper  is  to  build  a framework  for  evaluating  online  learning  environments  using 
activity  theory  as  a lens,  thereby  contributing  to  designing  more  authentic  online  learning  environments.  Activity 
theory  is  used  to  provide  an  analytical  lens  through  which  to  view  and  analyze  learners’  activities  and  interaction  in 
online  learning  environments.  The  framework  is  not  prescriptive,  meaning  that  it  would  not  tell  people  what  to  do  in 
a specific  circumstance;  rather,  it  analyzes  aspects  of  online  learning  environments  in  order  for  evaluators  to  better 
understand  online  learning  environments  (Robson,  2000).  Online  learning  developers  can  modify  the  framework  to 
fit  in  a particular  evaluation  project.  Further,  the  framework  can  be  served  as  a rubric  that  people  can  reference  when 
designing  online  learning  environments. 

This  paper  briefly  describes  socio-cultural  aspects  of  online  learning  environments.  Also,  activity  theory  and  a 
framework  for  evaluation  using  activity  theory  are  presented.  Implications  of  the  framework  for  future  research 
conclude  this  paper. 
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Socio- Cultural  Aspects  of  Online  Learning  Environments 

Although  the  transition  of  knowledge  from  teachers  to  students  used  to  be  considered  a good  way  to  instill 
students  with  facts  and  information,  it  is  now  regarded  as  impossible  for  teachers  to  feed  knowledge  to  students  in 
the  era  of  information.  Rather,  learning  is  viewed  as  a process  for  constructing  knowledge,  and  the  teachers’  role  has 
changed  to  guides  or  facilitators.  According  to  Mayer  (1999),  there  are  three  views  of  learning  and  instruction: 
learning  as  response  acquisition,  learning  as  knowledge  acquisition,  and  learning  as  knowledge  construction.  The 
third  view,  learning  as  knowledge  construction,  means  highlighting  “the  interaction  of  persons  and  situations  in  the 
acquisition  and  refinement  of  skills  and  knowledge”  (Schunk,  2004,  p.  287),  at  least  from  a constructivism 
perspective. 

Based  on  Marx’s  philosophy,  Vygotsky  was  one  of  modern  pioneers  in  socio-cultural  psychology. 
Vygotsky  emphasizes  the  social  environment  as  a facilitator  of  learning,  where  cultural  and  historical  aspects 
meditate  between  mind  and  consciousness  (Lerman,  2001;  Schunk,  2004).  Vygotsky  focuses  on  the  interaction  of 
interpersonal,  cultural-historical,  and  individual  factors  (Schunk,  2004)  as  the  key  to  human  learning  and 
development.  His  theory  maintains  that  learning  and  development  cannot  be  understood  or  explained  without  the 
interaction  within  the  culture.  Furthermore,  Vygotsky  presents  the  concept  of  a zone  of  proximal  development 
(ZPD)  to  describe  the  whole  process  of  learning,  explaining  the  relationship  between  the  individual  subject  and  his 
or  her  environments.  According  to  Engestrom  (1987),  the  ZPD  is  “the  distance  between  the  present  everyday  actions 
of  the  individuals  and  the  historically  new  form  of  the  societal  activity  that  can  be  collectively  generated”  (p.  174). 
The  main  idea  of  the  ZPD  is  that  humans  collectively  learn  through  social  interaction  in  a social  and  cultural 
environment  (Barab,  Evans,  & Baek,  1996).  Because  “the  ways  of  organizing  the  world  that  are  created  by  culture, 
within  a social  and  historical  context,  become  internalized  by  the  person  through  symbolic  matter”  (Lerman,  2001, 
p.  103),  people  internalize  the  culture  of  the  environment  during  the  learning  process  through  interaction. 

Unlike  the  view  of  traditional  psychology  regarding  learning  as  the  individual’s  cognitive  reconstruction 
(Lerman,  2001),  socio-cultural  roots  of  psychology  argue  that  it  is  more  appropriate  to  say  that  learners  construct 
their  knowledge  through  interaction  with  their  environments  than  learn  something  isolated  from  others.  Thus,  the 
focus  is  on  a learning  context  rather  than  on  individuals.  Green  and  Gredler  (2002)  note  that  because  social 
constructivists  view  “the  classroom  as  a community  charged  with  the  task  of  developing  knowledge”  (p.  57), 
knowledge  cannot  be  separable  from  activities  in  the  classroom.  Likewise,  constructivist  online  learning 
environments  are  designed  for  students  to  construct  their  knowledge  through  interaction  with  other  entities,  thus 
learning  cannot  be  appropriately  explained  without  viewing  the  whole  learning  environments.  Bonk  and 
Cunningham  (1998)  regard  learning  as  fundamentally  social  and  argue  that  it  is  derived  from  authentic  interaction 
with  others  in  a community.  Accordingly,  learning  is  naturally  and  meaningfully  influenced  by  social  interaction  in 
the  socio-cultural  contexts  in  which  students  are  involved  (Bonk  & Cunningham,  1998;  Jonassen,  1999;  Havens, 
2004).  Jonassen  (1999)  identifies  several  components  of  constructivist  learning  environments  including  problem- 
based  and  case-based  learning,  learning  resources,  cognitive  tools,  conversation  and  collaboration  tools,  and  social 
and  contextual  support  based  on  Vygotsky’s  socio-cultural  theory. 

Activity  Theory  as  a Framework  for  Evaluating  Online  Learning  Environments 

Activity  theory  is  an  international  and  multidisciplinary  framework  for  studying  different  forms  of  human 
praxis  that  interlinks  individual  and  social  levels  (Kuutti,  1996).  It  has  originated  from  classical  German  philosophy, 
the  works  of  Marx  and  Engels  and  the  Soviet  Russian  cultural-historical  psychologists  such  as  Vygotsky,  Leont’ev, 
and  Luria  (Engestrom,  1999).  In  socio-cultural  tradition,  “the  focus  is  on  mechanisms  that  make  people  act  in 
similar  ways  within  in  a given  context”  (Havnes,  2004,  p.  162).  Investigating  learning  outcomes  can  tell  us  whether 
or  not  instruction  is  successful.  However,  it  cannot  provide  us  with  the  reason  why  people  succeed  or  fail  to  learn  in 
a given  context.  To  view  and  analyze  learners’  activities  or  learning  in  online  learning  environments,  we  need  to 
thoroughly  look  at  the  learning  environments.  Activity  theory  has  been  found  to  provide  a holistic  framework 
through  which  to  design,  analyze  and  evaluate  online  learning  environments  (Jonassen  & Rohrer-Murphy,  1999; 
Hew  & Cheung,  2003;  Barab,  Schatz,  & Scheckler,  2004;  Haven,  2004).  For  the  reason,  activity  theory  is  used  to 
establish  a framework  to  analyze  online  learning  environments. 

Activity  Theory  as  an  Analytical  Lens 

Activity  theory  maintains  that  all  human  activities  are  mediated  by  a third  artifact  labeled  tool  or 
instrument.  The  central  notion  of  activity  theory  was  well  represented  in  the  Vygotsky’s  triangular  model  of  subject, 
object,  and  tool  (Vygotsky,  1978).  According  to  Vygotsky,  people  are  not  directly  interacting  with  the  environment. 
Instead  of  accepting  the  simple  behavioral  stimulus  and  response  relationship,  he  came  up  with  the  idea  of  cultural 
mediation  of  actions  (Engestrom,  2001).  The  environment  interacts  with  human  beings  through  its  mediating 
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artifact,  or  tool.  Because  “artifacts  are  the  product  of  human  history  and  culturally  specific”  (Lerman,  2001,  p.  95), 
the  cultural  and  historical  aspects  of  the  environment  are  likely  to  influence  human  interaction  and  ultimately  human 
development  and  learning. 

Following  Vygotsky’s  theory  and  work,  Leont’ev  developed  a framework  for  the  theory  of  activity  that 
“situates  behaviour  within  social  context,  via  three  levels  of  description:  activity  system,  action,  and  operation” 
(Tolmie  & Boyle,  2000,  p.  125).  The  activity  system  comprising  a subject,  an  object,  and  meditating  tools  provides  a 
lens  to  analyze  group  and  individual  behaviors.  A subject  pursues  an  object  by  using  tools  reflecting  the  culture.  At 
the  highest  level  is  activity,  which  “reaches  beyond  the  individual  and  is  part  of  a wider,  social  context”  (Havens, 
2004,  p.  163).  Actions  refer  to  individual  and  usually  conscious  behaviors  directed  at  specific  goals.  With  individual 
knowledge  and  skills,  subjects  take  actions  to  move  forward  to  the  overall  object.  Actions  comprise  several 
associated  operations.  Operations  are  the  most  basic  and  simplest  level  of  human  activity.  We  tend  to  behave 
without  any  conscious  intention.  Writing  a paper  is  a good  example  as  an  activity.  Then,  reading  reference  books 
and  articles,  outlining  the  paper,  and  writing  a paragraph  are  some  of  actions  toward  the  goal.  The  operation  would 
be  typing  a word,  drawing  a concept  map  before  outlining. 


Rules 


Tool 


Figure  1.  An  activity  system  (Engestrom,  1987). 

Adopting  Leont’ev’s  work,  Engestrom  (1987)  expanded  Vygotsky’s  triangular  model  and  completed  the 
activity  system  (see  Figure  1).  In  this  model,  he  acknowledged  the  contextualized  nature  of  activity  by  adding  rules, 
community,  and  division  of  labor  (Barab  et  al.,  1996).  For  example,  taking  a basic  introductory  Hyper  Text  Markup 
Language  (HTML)  course  constitutes  an  activity  system.  The  course  is  an  exclusively  online  learning  environment 
without  face-to-face  instruction.  The  overall  objective  of  the  course  is  to  apply  the  knowledge  and  skills  of  HTML 
in  developing  Web-based  educational  courseware  (object).  The  teacher  (subject)  uses  WebCT  (tool)  to  deliver  pre- 
recorded lectures  (tool),  to  provide  the  students  (subject)  with  a platform  where  they  can  interact.  There  are  students 
in  the  class,  other  experts  of  HTML  on  the  Web  (community),  and  several  course  requirements  (rule).  The  teacher 
wants  the  student  to  work  in  groups  to  produce  an  educational  courseware  (division  of  labor).  The  teacher  in  the 
course  assigns  weekly  tasks  to  each  student  for  helping  them  move  toward  the  course  object.  Some  of  the 
assignments  are  inserting  images  and  texts,  creating  links,  formatting  HTML  pages,  and  creating  and  modifying 
tables.  To  achieve  the  overall  object,  or  to  create  Web-based  educational  courseware,  the  learners  should  complete 
the  weekly  assignments  (actions).  Each  action  is  a unique  task  directed  at  the  course  object  and  comprises  several 
operations.  Creating  and  modifying  tables  requires  the  learners  to  manipulate  columns,  rows,  borders,  colors,  etc. 
(operations)  with  provided  resources,  communication  and  collaboration  tools,  and  others. 

To  evaluate  the  effectiveness  of  the  course  described  above,  evaluators  should  look  at  all  the  components 
that  form  the  activity  system  or  the  learning  environment  and  how  they  interact  with  others  within  the  system.  The 
fact  that  each  component  of  online  learning  environments  interacts  with  others  brings  about  the  complexity  of 
evaluation  of  it.  It  has  been  known  that  activity  theory  understands  this  complexity  (Russell,  2002).  The  complexity 
of  the  interrelationship  has  called  for  a new  framework  to  work  with.  Next,  a framework  for  evaluating  online 
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learning  environments  is  more  elaborated  based  on  activity  theory.  Each  component  in  the  activity  system  is 
described  and  how  they  interact  with  others. 

Components  of  Activity  System  for  Evaluation 

Six  components  are  identified  as  an  action  system  for  evaluation:  subject,  object,  tool,  rules,  community, 
and  division  of  labor. 

Subject.  A subject  refers  to  an  individual  or  groups  of  people  who  are  engaged  in  an  activity.  In  online 
learning  environments,  students  are  the  subjects.  Since  each  of  the  subjects  participates  in  many  other  activity 
systems  (e.g.,  school,  home,  work,  etc.),  it  is  critical  to  recognize  where  they  come  from  (Russell,  2002).  Students’ 
prior  knowledge  and  experiences,  learning  styles,  motivation,  and  attitudes  can  influence  learning  and  performance 
in  online  learning  environments. 

Object.  An  object  refers  to  a concrete  goal  that  an  activity  is  moving  toward;  that  is,  the  object  indicates  an 
overall  direction  of  that  activity  or  a shared  purpose  and  it  is  culturally  and  historically  formed  (Russell,  2002).  In 
online  learning  environments,  this  might  be  the  objectives  of  a course.  The  shared  object  is  critical  in  that  students 
can  act  toward  the  goal  and  give  meaning  to  actions  related  to  the  activity. 

Tool.  Tools  are  perceived  as  mediating  artifacts  between  subject  and  object.  They  include  both 
psychological  tools  such  as  language,  signs  and  symbols,  and  material  tools  like  pencils  and  paper  (Barab  et  al., 
1996).  In  online  learning  environments,  different  communication  technologies  can  be  mediating  tools,  such  as 
discussion  board,  chatting  room,  email,  etc.  It  seems  natural  that  using  different  delivery  tools  will  bring  about 
different  outcomes.  The  tools  that  people  use  and  the  variety  of  ways  they  use  the  tools  have  changed  over  time  as 
other  activity  systems  produce  new  tools  and  new  ways  to  use  them  (Russell,  2002).  For  example,  the  development 
of  technologies  enables  us  to  use  the  Web  as  a learning  environment. 

Rules.  Rules  refer  to  norms,  routines,  and  conventions  of  behaviors.  The  rules  mediate  between  subject  and 
community  and  also  shape  the  interactions  of  subject  and  tools  with  the  object  (Russell,  2002).Rules  in  online 
learning  environments  are  different  for  those  of  traditional  classroom  environments. 

Community.  According  to  Russell  (2002),  the  subject  is  a community  or  a part  of  a larger  community  and 
the  community  “conditions  all  the  other  elements  of  the  system”  (p.  70).  In  online  learning  environments,  they  often 
form  a learning  community. 

Division  of  Labor.  Activity  system  has  a division  of  labor.  A division  of  labor  refers  to  organization  of 
processes  related  to  the  goal.  In  the  process,  subjects  play  different  roles  in  the  activity.  In  traditional  face-to-face 
classroom  environments,  the  labor  is  divided  into  a teacher  and  students  (Russell,  2002).  However,  in  online 
learning  environments,  the  labor  can  be  divided  in  many  different  forms.  In  many  cases,  teachers’  roles  have 
changed  to  facilitators  and  students  form  a group  and  collaborate  among  themselves  to  achieve  the  overall  goal. 
Furthermore,  considering  the  complexity  resulting  from  the  interrelationship  among  the  components,  the  following 
sub-triangles  from  the  activity  system  are  useful. 

Triad  of  subject-tool-object.  Tools  mediate  between  subject  and  object.  This  triad  looks  at  how  effectively 
tools  help  the  subjects  get  to  the  goal.  What  tools  are  used  and  how  are  they  used  to  deliver  the  course  contents?  Do 
they  cover  all  knowledge  and  skills  necessary  for  the  subjects  to  obtain  the  goal?  If  the  tools  are  not  easy  to  follow, 
then  opportunities  for  performing  the  object  well  may  be  diminished. 

Triad  of  subject-rules -community.  Rules  mediate  between  subject  and  community.  The  triad  describes  how 
the  subjects  interact  with  others  in  the  learning  community.  How  do  subjects  understand  the  interaction  rules  of  the 
community?  This  is  particularly  difficult  in  online  learning  environments  where  there  is  no  face-to-face 
communication  to  clarify  and  negotiate  the  rules  (Russell,  2002). 

Triad  of  subject-tool-community.  Tools  mediate  between  subject  and  community.  If  insufficient 
communication  and  collaboration  tools  are  used,  creating  a community  and  keeping  the  community  moving  toward 
the  goal  may  be  challenging. 

Triad  of  subject-division  of  labor-community.  A division  of  labor  mediates  between  subject  and 
community.  What  roles  are  implied  to  students  in  the  community?  How  do  these  roles  contribute  to  the  community? 
If  students’  roles  in  the  community  are  not  clearly  identified,  opportunities  for  collaborating  are  likely  to  be 
diminished. 

Triad  of  subject -community -object.  In  this  triad,  a community  mediates  between  subject  and  object.  Do  the 
subjects  feel  a part  of  the  community  to  accomplish  the  goal  together?  If  the  object  is  highly  individualized,  then 
opportunities  for  collaborating  are  likely  to  be  diminished. 

Triad  of  object-division  of  labor-community.  A division  of  labor  mediates  between  object  and  community. 
The  triad  focuses  on  how  roles  of  the  community  contribute  to  accomplishing  the  goal  of  the  activity.  If  the 
instruction  is  designed  for  individual  work,  opportunities  for  collaboration  may  be  reduced. 
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Triad  of  object-tool-community.  Tools  mediate  between  object  and  community.  What  tools  are  used  for 
subjects  to  collaborate  and  cooperate  with  other  community  members  in  order  to  accomplish  the  goal?  If  tools  are 
not  sufficient  to  communicate  and  collaborate  with  others,  many  in  the  community  may  fail  to  learn. 

Triad  of  object-rules-community.  Rules  mediate  between  object  and  community.  If  there  is  no  explicit  rule 
for  working  together  in  online  learning  environments,  collaboration  in  the  community  is  unlikely  to  occur. 

Implications  for  Future  Research 

As  the  development  of  technologies  progresses,  people  want  benefits  from  online  learning  environments 
(Bonk,  2002).  The  number  of  students  taking  online  courses  has  been  rapidly  increasing  in  recent  years.  Given  the 
growing  number  of  learners,  there  are  many  concerns  with  how  to  create  online  learning  environments  supporting 
constructivists’  view  of  learning.  As  the  interests  in  online  learning  environments  from  constructivists’  viewpoints 
have  increased,  we  challenge  how  to  evaluate  the  learning  environments  as  a whole.  In  this  paper,  an  evaluation 
framework  using  activity  theory  is  presented.  Activity  theory  has  been  proved  to  be  a useful,  practical,  and 
analytical  lens  to  view  human  activities  or  learning  in  online  learning  environments.  The  components  of  the  activity 
system  are  described  and  some  possible  evaluation  questions  are  also  presented. 

The  activity  theoretical  framework  enables  us  to  look  at  the  entire  learning  environment,  describe  how 
every  operation  and  action  is  directed  to  the  goal,  and  how  they  interact  with  others.  An  area  of  future  research 
would  be  to  explore  why  substantial  number  of  students  decides  not  to  continue  to  take  online  courses  and  how 
some  survive  online  learning  environments.  Robson  (2000)  shows  that  discontinuation  in  online  learning 
environments  are  considerable.  Because  most  evaluation  models  for  online  learning  do  not  consider  these 
discontinuing  students  (Robson,  2000),  it  is  difficult  to  investigate  the  reason  why  they  decide  to  discontinue  taking 
an  online  course.  Analyzing  the  elements  of  online  learning  environments  using  the  framework  presented  in  this 
paper  can  give  us  a better  idea  of  how  to  design  effective  learning  contexts  where  students  actively  construct  their 
knowledge  through  interaction  with  others  and  resources.  Furthermore,  because  the  evaluation  of  learning 
environments  tends  to  be  summative  as  well  as  formative  evaluation,  this  framework  can  be  used  as  a guideline  for 
instructional  designers  to  reference  when  they  design  online  learning  environments.  Also,  it  could  provide 
pedagogical  guidance  for  teachers’  individual  instruction  activities. 
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Abstract 

The  study  focuses  to  investigate  the  use  of  technologies  for  supporting  virtual  teams.  Since  fast-emerging 
distance  educational  fields  have  adopted  a collaboration  approach  for  learning  purpose,  better  understanding  the 
nature  of  virtual  teaming  activities  is  required.  With  consideration  of  the  fact  that  virtual  teaming  activities  assist 
students  to  build  collective  knowledge  and  skills,  the  study  conceptually  connects  three  different  virtual  team  modes 
with  a category  of  appropriate  interactive  technologies.  Also,  this  study  briefly  addresses  the  current  use  of 
interactive  technologies  in  virtual  teaming  activities.  The  study  suggested  the  appropriate  integration  of  different 
technologies  for  optimal  virtual  teaming  activities. 


Introduction 

Teamwork  is  one  of  the  most  widely  used  instructional  activities  in  traditional  classrooms  as  well  as  online 
learning  environments.  Many  studies  point  out  that  teamwork  activities  provide  pedagogically  rich  context  to  assist 
students  in  building  meaningful  knowledge  in  online  environments  (Carabajal,  LaPointe,  & Gunawardena,  2003; 
Lee,  Bonk,  Magjuka,  Su,  & Liu,  in  press;  Palloff  & Pratt,  2005).  In  this  sense,  the  use  of  teamwork  activities  in 
online  environments  has  gained  more  competitive  advantages  compared  to  other  instructional  activities. 

When  teamwork  is  introduced  in  online  environments  (it  can  be  called  ‘virtual  teams’),  the  context  for 
virtual  teaming  activities  becomes  different  from  that  of  traditional  classroom  teamwork.  Virtual  team  members  are 
dispersed  in  physically  and  culturally  different  environments.  Any  idea  exchanges  and  decision  making  within  the 
teams  via  technologies  requires  more  time,  effort,  and  detailed  guidelines  than  would  be  required  for  regular 
classroom  teamwork. 

Technology  explosion  within  virtual  team  members  is  one  of  the  major  concerns  that  has  seldom  been 
considered  in  classroom  teamwork  situations.  Technologies  for  virtual  teams  function  as  communicational  channels, 
instructional  medium,  and  mutual  product  tools.  Team  members  adjust  themselves  to  get  accustomed  to  the  usage  of 
technological  tools  for  enriched  team  operation,  as  well  as  effective  ways  of  how  to  learn  together. 

The  success  of  virtual  teams  depends  on  the  appropriate  integration  of  technologies  with  pedagogical 
activities.  What  virtual  team  members  pursue  determines  the  modes  of  virtual  teams  and  the  use  of  technologies,  and 
they  eventually  bring  an  impact  on  the  virtual  team’s  process  and  performance.  However,  the  increased  use  of 
virtual  teaming  activities  has  not  been  accompanied  by  research  efforts  to  better  understand  interactive  technologies 
to  support  different  modes  of  virtual  teams  in  online  environments.  Thus,  this  study  covers  two  research  interests: 
first,  the  literature  review  conceptually  connects  three  different  virtual  team  modes  with  a category  of  appropriate 
interactive  technologies.  Secondly,  this  study  briefly  addresses  the  current  use  of  interactive  technologies  in  virtual 
teaming  activities.  An  understanding  of  different  interactive  technologies  under  different  modes  of  virtual  teams  will 
be  of  fundamental  interests  for  instructional  designers  and  practitioners. 

Different  modes  of  virtual  teams,  different  technologies 
Three  different  modes  of  virtual  teams 

Virtual  teams  can  be  defined  as  “a  group  of  people  with  complementary  competencies  executing 
simultaneous,  collaborative  work  process  through  electronic  media  without  regard  to  geographic  location” 
(Chinowsky  & Rojas,  2003,  pp.98).  Teamwork  for  educational  purpose  requires  a group  of  students  to  pursue  a 
common  learning  goal  through  completing  certain  tasks.  Virtual  teams  have  different  operation  mechanisms  or  ways 
of  interactions  than  traditional  classroom  teams.  Usually  classroom  teamwork  is  carried  out  by  individual  students 
who  are  physically  in  the  same  place.  Meanwhile,  virtual  team  members  reside  in  different  places  and  seek  to 
accomplish  interdependent  tasks  throughout  by  interactive  technologies. 

The  essential  nature  of  virtual  teams  is  the  growth.  Virtual  teams  start  “gathering  together  online”  at  the 
beginning,  and  gradually  develop  themselves  to  the  maturity  level  of  “learning  online  collaboratively”.  The  stage  at 


366 


which  virtual  teams  operate  shapes  team  behaviors  and  performance  in  the  end.  Under  the  modified  viewpoint  of 
Himmelman  (2004)’ s developmental  continuum,  virtual  team  can  be  distinguished  into  three  modes  based  on  what 
they  pursue  as  a team  and  how  they  behave:  communication  cooperation,  and  collaboration. 

Communicational  virtual  teams20  are  basic  forms  of  team  at  the  early  stage  of  team  development. 
Exchanging  information  and  changing  work  activities  for  mutual  benefits  are  key  concerns  to  team  members 
(Himmelman,  2004).  Sending  and  receiving  messages,  and  transmitting  information  among  team  members  via 
technologies,  are  ordinary  behaviors.  No  additional  data  processing  or  manipulation  beyond  communicational  needs 
are  transmitted  or  interpreted  (Chino wsky  & Rojas,  2003).  Consequently,  the  use  of  technological  tools  in  this  mode 
is  limited  to  message  delivery. 

Effective  communications  are  the  fundamental  bases  to  make  teamwork  environments  friendly,  especially 
when  they  do  not  meet  face-to-face.  However,  it  is  not  enough  to  simply  accomplish  team  goals.  Once 
communicational  behaviors  contribute  to  coordinate  good  relationships  among  members,  virtual  teams  evolve  into  a 
cooperation  mode.  Cooperative  virtual  teams  “exchange  information,  alter  activities,  and  share  resources,  for  mutual 
benefit  and  the  achievement  of  a common  purpose”  (Himmelman,  2004,  p.  2).  In  a cooperative  virtual  team  mode, 
team  members  divide  labor  and  take  responsibilities  for  a portion  of  the  mutual  tasks  (Lehtinen,  Hakkarainen, 
Lipponen,  Rahikainen,  & Muukkonen,  n.d.). 

On  the  other  hand,  collaborative  virtual  teams  involve  mutual  engagement  in  a coordinated  effort  to  solve 
problems  together  (Roschelle  & Teasley,  1995).  In  other  words,  they  focus  on  enhancing  the  capacity  of  another 
individual  or  organization  for  team  benefit,  and  on  achieving  a common  purpose  (Himmelman,  2004).  Collaboration 
seeks  team  members  to  experience  meaningful  learning  throughout  the  whole  process  of  producing  team  products. 

As  Himmleman  (2004)  suggested,  the  different  modes  of  teamwork  can  work  better  under  specific 
circumstances  depending  on  what  common  tasks  and  what  barriers  they  are  facing.  However,  considering  the 
features  of  online  environments,  what  technologies  are  being  used  and  how  technologies  can  support  teams  in 
different  modes  should  be  taken  into  consideration. 

Interactive  technologies  for  virtual  teams 

Students  select  technological  tools  when  their  tool  functions  are  understandable.  Students  use  technological 
tools  better  when  the  use  of  technological  tools  is  pedagogically  well  designed.  Pedagogical  design  of  technology 
use  means  that  online  instructors  and  educational  practitioners  carefully  integrate  the  use  of  technological  tools  into 
instructional  activities.  Proper  technology  integration  supports  building  meaningful  learning  experiences  without 
meeting  face-to-face.  In  addition,  Chinowsky  and  Rojas  (2003)  stressed  that  the  roles  of  technologies  in  virtual 
teaming  activities  go  beyond  electronic  communications,  transferring  information,  and  the  products  of  team  projects. 
In  a similar  sense,  Salomon,  Perkins,  and  Globerson  (1991)  mentioned  about  importance  of  the  use  of  technologies 
as  cognitive  tools.  What  this  means  is  that  students  are  aware  of  what  technological  tools  they  will  use  for  certain 
activities,  instead  of  being  overwhelmed  by  technologies  themselves.  By  doing  so,  the  use  of  technologies  as 
cognitive  tools  can  enhance  students’  thinking  and  learning. 

Interactive  technologies  for  virtual  teamwork  should  be  able  to  “communicate,  collaborate,  and  interact  to 
share  knowledge  and  information”  (Mayben,  Nichols,  & Wright,  2003;  Sole  & Applegate,  2000).  Currently  many 
current  course  management  systems  support  virtual  teaming  activities  with  advanced  technological  tools  containing 
a variety  of  functions.  However,  the  proper  application  of  technological  tools  into  the  virtual  teaming  activities  is  not 
fully  known.  It  can  be  taken  for  granted  that  the  appropriate  use  of  technologies  determines  the  quality  of  virtual 
teamwork  process  and  performance.  However,  many  questions  regarding  effective  use  of  those  technologies  remain 
unsolved.  For  example,  what  technological  tools  can  we  consider  for  each  of  virtual  team  modes  (communication, 
cooperation,  and  collaboration)?  How  can  we  use  the  technological  tools  effectively  as  cognitive  tools? 

Mayben,  Nichols,  and  Wright  (2003)  list  a variety  of  collaborative  technologies  including  email,  file 
transfer  protocol,  video  conferencing,  discussion  boards,  electronic  mailing  list,  instant  messaging  programs,  digital 
imaging,  Web  Sites,  phone/teleconferencing,  and  word  processing  editing  tools.  Also,  the  existing  studies  have 
mostly  used  technology  classification  of  synchronous  and  asynchronous  applications  (Coleman,  1997;  Durate  & 
Snyder,  1999;  Reynolds,  1994).  However,  since  this  study  examines  different  technologies  in  different  virtual  team 
modes,  it  utilizes  Chinowsky  and  Rojas  (2003)’ s spectrum  of  electronic  interaction  technologies.  This  classification 


20  Himmelman  (2004)  in  his  developmental  continuum  for  change  strategies  points  out  “collaboration  is  in  a relation 
to  three  other  strategies  for  working  together:  networking,  coordination,  and  cooperation  (p.  1).”  Instead,  this  study 
combines  “networking”  and  “coordination”  under  the  name  of  “communication”  in  order  to  clearly  distinguish  it 
from  the  other  two  modes  of  virtual  teams  (i.e.,  cooperation  and  collaboration). 
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of  technologies  is  useful  to  examine  the  actual  technology  use  in  three  modes  of  virtual  teams  (such  as 
communication,  cooperation,  and  collaboration). 

The  function  of  communication  technologies  is  to  deliver  messages  to  one  team  member  or  the  team,  and  to 
exchange  thoughts  under  discussions.  These  messages  can  be  asynchronous  or  synchronous.  Chinowsky  and  Rojas 
(2003)  point  out  that  phone/teleconferencing,  and  representative  traditional  communication  channels,  can  be  still 
used  for  virtual  teaming  activities  involving  communication. 

Cooperation  technologies  feature  technical  advances  over  communication  technologies.  The  central  focus 
of  cooperation  technologies  is  concerned  with  asynchronous  interactions  and  information  manipulation  for  team 
tasks.  Project  Web  sites  and  discussion  forums  fall  into  the  category  of  cooperation  technologies.  Virtual  team 
members  access  the  teamwork  space,  exchange  communications,  and  update  the  project  documentations.  However, 
the  limitation  of  cooperation  technologies  is  that  they  support  mostly  asynchronous  teamwork  and  do  not  go  beyond 
the  real-time  experiences. 

On  the  other  hand,  collaboration  technologies21  put  more  focus  on  real-time  functions  of  the  tools. 
Synchronous  communications  and  data  productions  are  critical  if  team  members  need  to  accomplish  mutual  team 
products  in  a limited  time.  Two  way  synchronous  tools  such  as  videoconferencing  or  web  conferencing  could  create 
similar  environments  with  face-to-face  virtual  teams.  Those  tools  can  support  virtual  teams’  efforts  to  real-time  data 
manipulation  and  idea  exchanges. 

Each  interactive  technological  tool  features  its  own  functions  and  supports  virtual  teaming  activities 
somehow  across  the  developmental  continuum  of  virtual  teams.  However,  this  study  points  out  that  there  is  a 
limitation  of  the  use  of  existing  technologies  to  support  the  team  member’s  effort  to  collaborate.  Importantly,  mere 
operation  of  virtual  teaming  activities  does  not  guarantee  the  effectiveness  and  efficiency  of  the  teamwork.  What  is 
more  important  is  to  identify  technologies  that  have  the  potential  to  improve  the  performance  of  virtual  teams. 

How  interactive  technologies  were  used  for  virtual  teams? 

This  study  examined  a well  known  online  MBA  program  as  a case  study.  Course  content  analyses,  faculty 
interviews,  and  student  surveys  were  used  in  order  to  analyze  the  use  of  interactive  technologies  for  virtual  teams. 
Different  data  sets  were  cross  checked  as  a means  of  triangulation  method  and  this  study  mainly  presents  a small 
piece  of  content  analysis  and  student  survey  due  to  the  page  limitation.  The  data  on  the  detailed  use  of  interactive 
technologies  that  have  been  utilized  for  virtual  teaming  activities  were  reassembled  into  the  categories  of 
communication,  cooperation,  and  collaboration. 

According  to  the  content  analysis,  twenty-one  out  of  twenty-seven  courses  used  virtual  team  activities  (78 
percent).  Surveys  pointed  out  students  are  relatively  satisfied  with  teamwork  in  online  environments.  In  detail,  they 
responded  that  online  courses  provided  them  with  useful  tools  for  collaboration  (71.6  percent,  M=3.74  SD=1.01). 
Overall,  students  agreed  that  virtual  teamwork  contributed  to  build  co-knowledge  among  themselves  (M=4.18, 
SD=.80). 

Use  of  communication  technologies : According  to  the  content  analyses,  most  virtual  teams  analyzed  in  this 
study  frequently  used  ordinary  communicational  channels.  The  eight  courses  used  phones/teleconferencing  (30 
percent)  and  twenty-six  courses  utilized  emails  (96  percent).  It  is  assumed  that  virtual  team  members  easily  selected 
those  technological  tools  for  communication  purpose  due  to  relatively  easy  access  and  functional  familiarity  of  the 
tools. 

Use  of  cooperation  technologies : The  results  of  course  content  analyses  showed  that  eight  courses  (30 
percent)  used  discussion  forums  for  virtual  teaming  activities.  Team  file  sharing  was  used  in  nine  courses  (33 
percent).  The  cooperation  technologies  function  for  asynchronously  creating,  modifying,  storing,  retrieving  team 
products.  Also,  team  discussion  forums  asynchronously  used  updating  ideas  among  team  members. 

Use  of  collaboration  technologies : The  usage  level  of  one-way  and  two-way  synchronous  tools  such  as 
chat,  instant  messaging,  or  video  conferencing  was  reviewed.  Five  courses  reported  using  chat  tools  for  discussion  in 
virtual  teams  (19  percent)  and  one  course  (4  percent)  used  LiveMeeting.  The  official  use  of  video  conferencing  and 
instant  messages  was  not  shown  in  the  online  courses  which  were  analyzed  in  this  study.  Providing  students  with  the 
chances  of  exchanging  ideas  synchronously  is  critical  to  solicit  team  members  to  negotiate  and  make  decisions.  The 
combined  use  of  asynchronous  and  synchronous  tools  can  help  team  members  better  deal  with  conflicts  that  usually 
occurred  in  the  middle  of  teamwork  process  due  to  little  presence  of  team  members. 

Overall,  the  review  of  interactive  technologies  for  virtual  teaming  activities  shows  that  many  of 
technological  tools  are  embedded  in  the  course  management  systems  were  in  effective  use.  In  practice,  most  virtual 


21  Chinowsky  and  Rojas  (2003)  classified  virtual  teaming  into  interactive  technologies  of  collaboration.  However, 
this  study  does  not  see  virtual  teaming  as  technological  tool  itself,  but  it  is  an  intentional  instructional  activity. 
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teams  remain  in  heavy  use  of  both  communication  and  cooperation  technologies.  The  actual  use  of  collaboration 
technologies  recorded  as  quite  low,  compared  to  the  other  two  categories  of  technologies.  It  is  natural  that  virtual 
team  members,  at  the  beginning,  start  to  work  together  in  a communication  mode  and  heavily  rely  on  emails  and 
phone  talks  for  facilitating  communications.  However,  communication  behaviors  and  technologies  have  constraints 
to  support  producing  team  deliverables  and  do  not  always  improve  their  higher-order  thinking  and  knowledge- 
building. Teamwork  for  learning  purposes  should  go  beyond  information  transmission,  labor  division,  and  working 
independently. 

Kirschner  and  Van  Bruggen  (2004)  stress  the  point  that  from  a pedagogical  point  of  view,  current 
interactive  technologies  for  teamwork  are,  generally  speaking,  designed  for  mechanical  functions  of  basic 
communications,  or  limited  use  for  supporting  teaming  activities.  Similarly,  simple  implementation  of  virtual 
teaming  activities  does  not  guarantee  the  effectiveness  and  efficiency  of  the  teamwork.  It  is  more  important  to  guide 
appropriate  usage  of  technological  tools  to  support  the  whole  process  of  collaborative  learning  so  that  it  can  be 
eventually  leading  virtual  team  members  to  improve  the  team  performance. 

Summary 

The  nature  of  team  tasks  and  modes  of  team  development  determine  team  behaviors.  In  particular,  what 
team  members  are  pursuing  influences  the  selection  of  interactive  technologies  for  virtual  teaming  activities. 
Consequently,  the  use  of  technologies  impacts  the  quality  and  depth  of  virtual  teaming  process  and  performance. 

In  this  study,  the  use  of  different  technologies  to  support  virtual  teaming  activities  was  investigated. 
Kirschner  and  Van  Bruggen  (2004)  have  raised  issues  that  current  technologies  put  more  weight  on  so  called 
“functional  collaboration,”  not  on  “conducting  learning  tasks  properly,  understanding  each  other’s  work  and 
teamwork  process.”  What  they  point  out  is  the  use  of  many  technological  tools  remains  facilitating  communicational 
behaviors  such  as  message  delivery,  but  not  assisting  building  collective  knowledge. 

In  practice,  this  study  confirms  that  virtual  teaming  activity  as  an  instructional  method  has  been  positively 
used  in  online  learning  environments.  However,  the  practices  of  virtual  teaming  activities  still  rely  on  technologies 
of  communication  and  cooperation,  rather  than  collaboration  technologies.  The  introduction  of  technologies  in 
virtual  teaming  activities  itself  do  not  assure  meaningful  collaborative  learning  experiences.  With  consideration  of 
this  fact,  elaborated  design  of  appropriate  technologies  into  virtual  teaming  activities  are  required.  While  this  study 
focuses  the  relationship  of  interactive  technological  tools  with  collaborative  learning,  the  focus  of  next  study  is 
expected  to  develop  practical  guidelines  to  use  technologies  for  different  stages  of  the  collaborative  learning 
process. 
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Demobank:  a method  of  presenting  just-in-time  online  learning 
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University  of  South  Florida 

Abstract 

A demobank  is  a method  of  presenting  web-based  instruction  for  promoting  procedure-based  learning.  This 
presentation  technique  is  a well  structured  collection  of  web-based  animated  demonstrations  ( narrated 
demonstrations  of  computer-based  procedures ).  A demobank  allows  the  learner  to  view  animated  demonstrations 
individually  or  as  a series.  These  demonstrations  usually  depict  a group  of  computer-based  procedures,  which  if 
properly  constructed  can  act  as  a help  system  or  as  a “just-in-time”  online  workshop.  Animated  demonstrations  are 
increasingly  being  used  by  a number  of  companies  to  deliver  web-based  training  to  their  intended  audiences,  either 
potential  clients  or  employees.  This  paper  discusses  learning  given  a demobank  of  animated  worked  examples  and 
design  considerations  for  this  new  presentation  technique. 

Animated  worked  examples? 

Many  people  believe  they  must  first  perform  a procedure  to  learn  that  procedure,  sometimes  repeatedly. 
However,  one  can  learn  how  to  perform  a procedure  just  by  studying  examples  (Sweller  and  Cooper,  1985). 
Certainly  the  correct  performance  of  a procedure  reinforces  learning,  but  as  Sweller  and  Cooper  clearly  demonstrate, 
performance  is  not  truly  necessary  to  learn  a procedure.  They  found  that  learners  that  studied  worked  examples 
required  significantly  less  time  during  skill  acquisition,  and  that  these  learners  made  significantly  fewer  errors 
during  problem  solving.  This  paper  builds  upon  Sweller’ s worked  examples  studies  to  produce  a just-in-time 
learning  environment  which  is  generalizable  to  all  computer  based  applications. 

For  decades  researchers  have  known  that  graphics  are  necessary  in  instruction,  because  they  support  verbal 
communication  (Bransford  and  Johnson,  1972).  More  recently  it  has  been  shown  that  learners  using  multimedia 
significantly  out-performed  learners  using  single  media  instruction  (Briinken,  Steinbacher,  Plass,  and  Leutner,  2002; 
Mayer  and  Moreno,  1998;  Mousavi,  Low,  and  Sweller,  1995).  Mousavi,  Low  and  Sweller  (1995)  suggested  that  the 
simultaneous  use  of  visual  and  verbal  media  effectively  increases  the  learners’  working  memory  capacity.  Briinken 
et  al.  (2002)  suggested  that  this  is  because  multimedia  makes  use  of  the  combined  resources  of  both  the  visual  and 
auditory  subsystems  of  working  memory. 

Chandler  and  Sweller  (1992)  succinctly  define  worked  examples  as  “a  problem  statement  and  the 
appropriate  steps  to  solution  (Chandler  and  Sweller,  1992,  p.  233).”  Animated  demonstrations  clearly  fall  under  this 
category  as  they  describe  a problem  and  its  solution  in  a series  of  steps.  Because  animated  demonstrations  are  goal 
directed  and  procedure-based  they  act  as  animated  worked  examples. 

Several  researchers  have  studied  the  instructional  effectiveness  of  animated  demonstrations  (Palmiter, 

1991;  Harrison,  1995;  Lipps  et  al.,  1998;  Waterson  & O’Malley,  1993).  Like  Sweller,  these  researchers,  found  that 
this  form  of  instruction  speeded  skill  acquisition,  and  that  learners  were  faster  and  more  accurate  when  learning 
procedural  tasks.  Sweller  suggests  this  is  because  these  learners  concentrate  on  relevant  problem  states  and  the  steps 
necessary  to  solve  problems  (Sweller  et  al.,  1998).  In  addition  performance  may  be  improved  because  of  the 
multimodal  nature  of  this  form  of  instruction. 

Given  such  a useful  form  of  instruction,  how  might  we  improve  upon  this  presentation  technique?  Sweller 
suggests  instructional  designers  should  design  instruction  with  the  human  cognitive  architecture  in  mind  (Sweller, 
1993,  1999).  The  remainder  of  this  paper  explores  how  Sweller’ s statement  can  be  applied  to  the  design  of  a 
demobank  of  animated  worked  examples. 

How  learning  occurs  given  animated  worked  examples 

A distinction  should  be  made  between  procedural  and  declarative  knowledge.  Declarative  knowledge  refers 
to  mental  representations  of  the  outside  world;  whereas  procedural  knowledge  refers  to  processes  that  operate  on 
those  mental  representations  (van  Merrienboer,  1997).  Declarative  knowledge  is  primarily  concept  or  language 
based  whereas  procedural  knowledge  is  usually  goal  directed  and  aimed  at  “knowing  how.”  Demobanks  may 
describe  declarative  information  to  learners,  but  are  primarily  useful  for  teaching  learners  procedural  knowledge. 
Given  complex  cognitive  skills  require  both  declarative  and  procedural  knowledge  (van  Merrienboer,  1997),  a 
demobank  becomes  an  ideal  medium  for  teaching  these  complicated  skill  sets. 
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Sweller  and  Cooper  (1985)  suggested  worked  examples  were  useful  mainly  in  the  initial  stages  of  skill 
acquisition.  Researchers  have  been  describing  the  phases  of  skill  acquisition  for  over  forty  years  (Fitts,  1964;  Anzai 
and  Simon,  1979;  Anderson,  1983).  Anderson  continued  in  this  tradition  when  he  described  cognitive  skill 
acquisition  (Anderson,  1982).  In  Anderson’s  ACT  framework  (Anderson,  1976,  1983,  1993)  the  three  stages  of  skill 
acquisition  are  the  declarative  stage,  the  knowledge  compilation  stage,  and  then  finally  a procedural  stage 
(Anderson,  1983).  Since  animated  worked  examples  are  most  useful  early  in  skill  acquisition,  this  paper  will  strictly 
focus  on  learning  during  the  declarative  stage. 

During  the  declarative  stage,  learners  mentally  rehearse  procedures  during  a performance  (Anderson, 

1982).  Anderson  suggests  it  is  even  common  to  observe  learners  in  this  phase  verbally  rehearsing  the  steps  of  a 
procedure  (Anderson,  1983).  Performance  during  this  first  stage  although  laborious  is  often  poor  and  error  prone 
(Shiffrin  and  Dumais,  1981).  This  is  because  learners  must  first  understand  under  what  conditions  a particular 
procedure  is  used.  In  addition  they  may  also  be  considering  the  current  problem  state,  the  end  goal  and  differences 
between  the  two  (Ward  and  Sweller,  1990).  So  it  is  understandable  that  novice  learners  may  have  some  difficulty. 

Given  Anderson’s  framework,  the  steps  of  a procedure  are  worked  out  according  to  procedure  specific 
production  rules.  Production  rules  involve  if  then  statements  like  the  following: 

IF  the  goal  is  to  communicate  with  someone  in  another  state 

THEN  dial  the  telephone. 

A grouping  of  production  rules  is  called  a production.  Productions  are  similar  to  what  a behaviorist  would 
have  called  a stimulus-response  pair,  but  from  a cognitive  perspective  (Anderson,  1983),  because  it  takes  into 
account  decision  making.  So  procedural  knowledge  according  to  Anderson’s  framework  is  initially  represented 
verbally  in  working  memory,  and  then  later  during  many  hours  of  practice  converted  into  the  fluid  movements  of  an 
expert. 

Chi  et  al.  (1981)  noticed  that  experts  analyze  problems  differently  than  novices.  These  experts  gather 
information  to  categorize  problems  usually  according  to  schema  specific  rules  (Chi  et  al,  1981).  Once  a problem  is 
categorized,  the  expert  may  use  a set  of  schema  specified  production  rules  to  solve  that  particular  problem  (Chi  et  al, 
1981).  Chi  and  Anderson  are  working  from  complementary  theory  bases.  Chi  tends  to  place  more  emphasis  on 
categorization.  In  summary,  a problem  schema  is  basically  a set  of  production  rules  that  guides  a problem  solver  to 
first  categorize  a problem,  and  then  to  solve  that  problem,  by  following  the  production  rules  for  that  specific 
problem  category. 

Without  the  guidance  of  a demobank,  novice  learners  may  use  weak  methods  like  “means  ends  analysis  to 
try  to  solve  a problem  (Owen  and  Sweller,  1985;  Anderson,  1987).  Being  that  a demobank  of  animated 
demonstrations  is  a collection  of  animated  worked  examples  the  literature  on  learning  by  studying  examples  will  be 
described  in  this  next  section. 

Studying  worked  examples  versus  means-ends  analysis 

Owen  and  Sweller,  (1985)  found  that  most  novices  try  to  solve  problems  by  attempting  to  achieve  a series 
of  sub  goals.  This  strategy  is  called  “means  ends  analysis.”  Their  results  suggested  that  this  strategy  “places  a heavy 
load  on  cognitive  processing  capacity  (Owen  and  Sweller,  1985,  p.272).”  This  is  perhaps  Sweller’ s first  reference  to 
cognitive  load.  After  making  this  statement  John  Sweller  went  on  to  describe  cognitive  load  theory  (Sweller,  1988). 

Sweller  developed  cognitive  load  theory  as  a means  of  explaining  the  performance  of  novices  during 
schema  acquisition.  He  proposed  that  during  the  initial  stages  of  learning,  learners  are  bombarded  by  information, 
which  if  not  properly  regulated,  can  quickly  overload  a learner’s  limited  “working  memory”  and  thus  deteriorate 
performance  (Sweller,  1988).  Cognitive  load  theory  contends  that  if  instructional  designers  limit  the  amount  and 
complexity  of  information  impacting  learners  that  this  will  promote  more  efficient  schema  acquisition  and 
automation  (Sweller,  1988,  1993,  1994). 

Means-ends  analysis  is  like  trying  to  get  through  a maze  by  going  in  what  seems  to  be  the  best  direction  at 
the  moment.  Learners  using  this  strategy  often  make  errors,  given  the  maze  analogy  they  may  try  the  wrong  path 
toward  their  eventual  goal.  Many  flounder  and  may  find  themselves  going  in  circles.  Errors  like  these  may  cause  the 
learner  a great  deal  of  frustration. 

The  use  of  worked-examples  on  the  other  hand,  decreases  cognitive  load  by  presenting  only  enough 
information  to  learn  how  to  solve  a problem.  In  terms  of  the  maze  metaphor,  learning  with  worked  examples 
provides  the  learner  with  guidance  that  can  shorten  their  route  toward  learning.  In  many  cases  learners  learning  this 
way  as  opposed  to  learning  by  traditional  problem  solving  (via  means  ends  analysis)  required  less  time  to  study  the 
examples  and  then  later  less  time  to  solve  problems  (Sweller  and  Cooper,  1985).  Ward  and  Sweller  (1990)  explain 
means  ends  analysis  this  way:  “A  heavy  cognitive  load  is  imposed  because  of  the  need  to  simultaneously  consider 
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and  make  decisions  about  the  current  problem  state,  the  goal  state,  differences  between  states,  and  problem  solving 
operators  that  can  be  used  to  reduce  such  differences  (Ward  and  Sweller,  1990,  p.3).” 

Learners  using  means  ends  analysis,  often  make  errors  and  waste  time  trying  to  solve  problems,  rather  than 
using  that  time  effectively  to  learn  the  underlying  problem  schema.  Later  because  of  these  errors  those  using  means 
ends  analysis  may  have  difficulty  piecing  together  the  underlying  steps  of  the  overall  procedure. 

Zhu  and  Simon  (1987),  later  went  on  to  replicate  Sweller  and  Cooper’s  findings  with  a group  of  high 
schools  students  that  were  learning  algebra  and  geometry.  They  were  able  to  show  that  learners  using  worked 
examples  could  complete  a 3 year  curriculum,  in  only  2 years  — a full  year  ahead  of  their  peers  who  had  learned 
through  a traditional  problem  solving  curriculum.  Zhu  and  Simon  (1987)  also  found  that  this  group  of  students 
scored  slightly  higher  on  national  achievement  tests  as  compared  with  their  problem  solving  peers. 

The  results  from  these  early  studies  (Sweller  and  Cooper,  1985;  Cooper  and  Sweller,  1987,  Zhu  and  Simon, 
1987)  are  encouraging,  but  somewhat  counter  intuitive;  and  certainly  they  are  not  without  their  critics  (e.g.  Gabrys 
et  al.,  1993).  This  is  because  educators  have  had  “learning  by  doing”  ingrained  in  educational  culture,  since  the  time 
of  Dewey  (Kilpatrick,  1918).  But  Sweller’ s empirical  results  still  suggest  learners  using  worked  examples  learn  to 
perform  those  skills  quicker  and  with  fewer  errors  (Sweller  and  Cooper,  1985).  Why  might  this  be  the  case? 

Chi  et  al.  (1981)  compared  experts  and  novices  as  they  solved  physics  problems.  They  found  that  experts 
solve  problems  based  upon  solution  mode.  Novices  on  the  other  hand  concentrated  on  the  surface  features  of  the 
problems.  By  presenting  novices  with  worked  examples  instructional  designers  teach  them  to  be  more  like  experts. 
That  is  they  are  taught  to  categorize  problems  according  to  solution  mode.  This  is  as  compared  to  traditional 
problem  solving  techniques  which  expect  learners  to  grasp  the  means  to  a solution  while  being  simultaneously 
burdened  with  the  task  of  solving  the  problem.  Traditional  problem  solving  adds  cognitive  load  to  an  already 
demanding  situation. 

While  studying  learners  using  worked  examples,  Chi  et  al.  (1989)  shed  some  additional  light  on  the  worked 
example  effect.  They  suggested  students  learn  via  self-explanations.  This  supports  Anderson  framework  of  verbal 
explanation  of  problems  during  schema  acquisition  (1982).  Chi  et  al.  (1989)  monitored  students  using  think  aloud 
protocols.  They  found  that  “poor”  students  do  not  generate  sufficient  self  explanations.  “Good”  students  on  the  other 
hand  monitor  their  understanding  and  misunderstanding.  In  doing  so  they  were  better  able  to  locate  areas  of 
difficulty  and  concentrate  on  those  areas.  Unfortunately  “poor”  students  were  unable  to  locate  problem  areas  and  in 
turn  had  difficulty  when  the  time  came  to  perform. 

Practice  of  course  is  important  after  initial  learning  and  indeed  this  is  what  Zhu  and  Simon  (1987)  did  in 
their  longitudinal  study.  They  initially  taught  students  using  worked  examples  so  that  they  would  learn  to  categorize 
problems  based  upon  solution  mode.  Then  once  learners  could  categorize  problems  effectively  they  provided  them 
with  practice  solving  problems.  This  allowed  them  to  automate  their  skills  (Schneider  and  Shiffrin,  1977;  Shiffrin 
and  Schneider,  1977)  and  promoted  what  Bloom  later  described  as  “automaticity  (Bloom,  1986).” 

Implications  for  demobank  design 

The  problems  an  instructional  designer  can  provide  via  a demo  bank  are  diverse.  As  mentioned  earlier  this 
presentation  technique  is  generalizable  to  all  computer  based  instruction.  An  instructional  designer  must  make 
several  important  decisions  about  how  to  present  instruction  via  this  form  of  instruction.  Therefore  this  section  will 
apply  the  preceding  discussion  to  the  practice  of  demobank  design  and  provide  a set  of  demobank  design  guidelines. 
But  first  an  example  will  be  provided  to  give  some  context  to  these  guidelines. 

A demobank  was  developed  by  the  author  to  produce  a series  of  web-based  animated  worked  examples  for 
university  faculty  (note  figure  1).  Our  campus  PeopleSoft  implementation  (PantherSoft)  became  the  subject  matter 
for  this  tutorial.  The  menu  is  designed  to  support  the  schema  behind  this  software.  For  each  learner  action  a menu 
item  opens  an  animated  worked  example. 

The  initial  implementation  of  this  project  became  very  useful  as  a tool  to  teach  large  numbers  of  people  at 
their  own  convenience.  Because  this  multimedia  learning  environment  is  posted  on  the  web  it  is  available  24  hours  a 
day  7 days  a week.  Given  that  a demobank  is  posted  on  the  web,  learners  can  make  use  of  it  wherever  they  are  as  a 
just-in-time  online  workshop.  In  addition  it  is  also  available  to  learners  for  review,  as  they  solve  problems.  The 
PantherSoft  tutorial  is  just  one  example  of  a demobank,  which  can  be  generalized  to  form  a general  classification  of 
tutorials.  Several  design  issues  were  found  to  be  apparent  during  the  development  of  the  above  example.  These  are 
documented  here  and  can  serve  as  guidelines  for  future  designers. 

First  and  foremost  it  is  very  important  for  designers  to  work  with  experts  in  order  to  categorize  problems  by 
solution  mode.  Expert  users  can  suggest  how  a task  can  be  broken  down  into  constituent  sub  skills.  Given  that 
designers  arrange  worked  examples  according  to  how  experts  view  problems,  learners  will  begin  to  view  the 
software  like  experts. 
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Providing  smaller  segments  will  be  beneficial  to  the  learner  in  two  ways.  First  it  prevents  working  memory 
overload  by  decreasing  the  amount  of  information  impacting  the  learner.  In  addition  it  will  be  easier  on  the  learner’ s 
computer  because  smaller  video  segments  require  less  bandwidth.  Another  bandwidth  consideration  is  the  use  of 
audio.  Audio  does  require  more  bandwidth,  but  resist  the  temptation  to  remove  the  audio  from  the  presentation.  As 
mentioned,  earlier  multimodal  instruction  (simultaneous  use  of  visual  and  verbal  media)  has  been  shown  to  promote 
learning  because  it  makes  more  effective  use  of  a learners  working  memory  (Mousavi,  Low  and  Sweller,  1995). 

As  Gagne  (1965)  explained  many  years  ago  the  sequence  of  instruction  is  very  important.  It  would  be 
useful  to  list  animated  demonstrations  in  order  if  possible.  Be  clear  why  these  procedures  matter  and  explain  under 
what  conditions  a particular  procedure  is  used  and  why  the  order  or  sequence  of  tasks  is  important.  Perhaps  consult 
the  advice  of  an  expert  to  get  a better  understanding  of  the  sequence  of  tasks. 

Remember  Chandler  and  Sweller  described  a worked  example  as  “a  problem  statement  and  the  appropriate 
steps  to  solution  (Chandler  and  Sweller,  1992,  p.  233).  Therefore  it  is  important  to  explain  the  goal  of  the  problem  at 
hand.  This  confirms  for  the  learner  that  they  are  watching  the  appropriate  presentation.  In  addition  explain  the 
context  of  the  current  presentation  as  it  relates  to  others  in  the  demobank.  This  will  provide  the  learner  with  the 
appropriate  context. 

Learners  may  wish  to  return  to  your  tutorial  on  another  occasion.  For  this  reason  it  is  important  to  make 
these  demonstrations  available  as  just-in-time  instruction.  Designers  can  suggest  in  the  narration  that  learners  review 
certain  segments  in  the  presentation.  So  suggest  repetition  and  practice.  Demobanks  are  a good  way  to  present 
procedures  to  the  novice,  but  without  practice  they  will  remain  novices.  The  easiest  way  to  make  these 
demonstrations  available  as  just-in-time  instruction  is  by  publishing  them  as  a web  site,  complete  with  a menu  of 
tasks  based  on  the  problem  schema. 

The  development  of  animated  demonstrations  presents  a different  set  of  problems  from  most  video 
production.  During  most  video  production,  camera  operators  capture  the  simultaneous  movement  and  audio  of 
actors.  This  typically  occurs  through  many  takes  that  can  later  be  seamlessly  edited  together  to  produce  one 
continuous  stream  of  video.  However  this  is  not  as  easily  accomplished  with  screen  capture  software.  Even  though  it 
is  possible  to  record  audio  and  video  simultaneously,  this  is  not  suggested.  Screen  capture  software  products  capture 
on-screen  motion  but  it  is  suggested  that  this  is  only  be  integrated  with  audio  segments  in  post  production. 

Therefore  the  most  efficient  way  to  produce  animated  demonstrations  is  to  begin  with  the  narration. 
Narration  begins  as  a well  written  script.  Spend  time  carefully  considering  your  instructional  message  for  here  is 
where  much  of  the  instruction  takes  place.  Once  you  have  your  script  ready,  have  your  voice  talent  practice  the 
script  once  or  twice  before  recording  the  narration.  Narration  can  then  be  recorded  as  digital  audio  files  in  a 
computer.  Several  takes  may  be  necessary  to  record  the  complete  script.  This  audio  file  can  then  be  edited  to 
produce  one  continuous  error  free  file.  This  narration  file  can  then  be  played  back  to  cue  someone  as  they  capture 
the  video  component  of  demonstration  with  screen  capture  software  (e.g.  Techsmith  Camtasia  or  Macromedia 
Captivate).  This  should  also  be  rehearsed  several  times  before  making  a video  capture.  Next  a developer  can 
synchronize  the  audio  and  video  tracks  to  produce  a fluid  animated  demonstration.  Finally  these  movies  should  be 
saved  as  a steaming  file  format,  most  developers  prefer  Macromedia’s  Flash  file  format  because  of  its  ubiquity  and 
its  cross-platform  compatibility.  These  movie  files  should  be  displayed  on  a web  page  one  at  a time.  In  addition  each 
page  will  contain  a schema-based  menu  system  that  allows  the  learner  to  view  files  one  at  a time  at  their  own  pace. 

Summary 

Sweller  suggests  instructional  designers  should  design  instruction  with  the  human  cognitive  architecture  in 
mind  (Sweller,  1993,  1999).  Given  schema  theory,  an  ideal  demobank  would  exactly  mirror  an  expert’s  problem 
schema.  One  may  think  of  a demobank  of  animated  worked  examples  as  an  externalized  problem  schema.  Given  this 
conception  of  a schema,  presenting  a demobank  to  a novice  learner,  can  be  thought  of  as  “schema  sharing.”  This  is 
because  this  presentation  technique  allows  the  designer  to  group  animated  examples  according  an  expert  schema. 
Learners  can  use  this  built  in  organization  to  constructively  group  procedures  into  problem  categories  that  can  be 
used  to  solve  problems  according  to  procedures  outlined  by  the  animated  worked  examples.  Finally  once  a schema 
has  been  incorporated  into  the  learner’ s long  term  memory,  perhaps  through  the  performance  of  the  demonstrated 
procedure,  that  schema  is  successfully  shared  with  a learner. 
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This  developmental  study  documents  the  systematic  design  and  development  of  embedded  self-regulation 
(SR)  support  into  existing  instruction;  this  investigation  encompassed  two  common  aspects  of  developmental 
research,  how  a model  guides  the  design  process  and  what  features  facilitate  its  usability  (Driscoll  & Dick,  1999).  If 
the  SR  support  activities  are  systematically  designed  for  replicable  instruction,  SR  support  could  remove  much  of 
the  SR  planning  and  resource  coordination  effort  from  the  learner  and  instructor  and  place  it  on  the  instructional 
design.  The  instructors  may  be  able  to  incorporate  a prescribed  set  of  embedded  SR  activities  instead  of  an 
individualized  prescription  for  each  learner  (Ley  & Young,  2001). 

This  development  study  investigated  how  SR  may  be  designed  and  developed  within  instruction  and  the 
relationship  of  the  process  to  the  efficient,  effective  and  usable  instructional  support.  A review  and  analysis  of  SR 
models  initiated  the  study  and  addresses  model  usability  and  model  comparisons,  characteristics  of  Type  2 
developmental  research  (Richey,  Klein,  & Nelson,  2004),  the  analytical  framework  for  this  study.  The  theoretical 
foundation  for  this  developmental  study  places  SR  in  the  context  of  instructional  design  and  establishes  the 
relevance  of  instructional  SR  support,  consistent  with  Type  2 research  (Richey,  Klein,  & Nelson,  2004).  The  implicit 
instructional  design  and  development  requirements  and  constraints  of  each  SR  model  form  the  basis  for  comparing 
alternative  model  feasibility  and  usability.  Some  of  these  constraints  and  demands  preclude  certain  instructional 
design  or  development  conditions  and  create  substantial  barriers  to  others,  thus  providing  a rationale  for  selecting  a 
model  based  on  usability. 


SR  Model  Analysis  and  Selection 

We  identified  fourteen  SR  models  from  published  approaches  to  explaining  SR  in  a learning  context.  The 
models  include  a seminal  SR  model  (Corno  & Mandinach,  1983)  and  thirteen  others  published  between  1995  and 
2001.  The  term  model  refers  to  any  set  of  propositions  suggested  to  describe  self-regulation  processes  whether  or 
not  the  authors  identified  the  propositions  as  a model.  Three  content-dependent  models  were  excluded  from  the 
analysis  (Como  & Randi,  1999;  Maitland,  2000;  Pressley,  El-Dinary,  Wharton-McDonald,  & Brown,  1998).  Two 
variations  of  a similar  model  were  analyzed  as  one  (Winne  & Hadwin,  1998;  Winne,  1997).  The  ten  models  (Butler, 
1998;  Corno  & Mandinach,  1983;  Ertmer  & Newby,  1996;  Graham,  Harris,  Troia,  2000;  Ley  & Young,  2001; 
Manning,  Glasner,  & Smith  1996;  McCaslin  & Hickey,  1998;  McCombs,  2001;  Winne  & Hadwin,  1998; 
Zimmerman,  1998b)  included  in  the  analysis  were  both  prescriptive  and  descriptive. 

Several  models  have  been  developed  for  specific  learning  environment  conditions  or  requirements  in  part 
because  the  assumptions  of  an  SR  support  approach  have  been  tailored  to  specific  learning  environment  conditions 
and  requirements,.  Their  specificity  potentially  inhibits  their  adaptability  to  learning  environments  with  different 
conditions  or  requirements.  For  example,  SR  approaches  that  complement  instructional  content  models  (Butler, 

1998;  Graham,  Harris,  & Troia,  1998;  Ley  & Young,  2001;  Manning,  Glasner,  & Smith,  1996;  Winne,  1997)  should 
be  more  adaptable  to  a wider  range  of  instructional  content  than  content-dependent  SR  approaches  (e.  g.,  Corno  & 
Randi,  1999)  or  skill- specific  approaches  such  as  reading  (Maitland,  2000;  Pressley,  El-Dinary,  Wharton-McDonald, 
& Brown,  1998). 

Instructional  content  is  one  of  several  factors  that  may  preclude  some  SR  support  approaches.  Some  SR 
support  approaches  have  been  developed  for  specific  learner  populations  and  content  which  also  constrains  where 
and  how  SR  support  may  be  provided  within  instruction.  The  strategic  content  learning  model  (SCL)  for  basic  skills 
(Butler,  1998)  and  the  SR  strategy  development  model  (SRSD)  for  writing  skills  (Graham,  Harris,  & Troia,  1998) 
address  the  cognitive  and  affective  needs  of  learning-disabled  students. 

Another  factor  affecting  the  utility  of  an  SR  approach  is  the  role  of  an  instructor  or  tutor  in  an  SR  support 
approach.  Some  SR  support  approaches  implicitly  assume  an  instructor  physically  present  in  the  learning 
environment  and  include  instructor-led  or  instructor-guided  learning  conditions  (c.f.,  Butler,  1998;  Corno  & Randi 
1999;  Graham,  Harris,  & Troia,  2000;  Maitland,  2000;  Manning,  Glasner,  & Smith,  1996;  McLoughlin,  1996). 
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Teacher-directed  activities  offer  SR  support  to  compensate  for  weak  self-regulation  characterized  by  insufficient  or 
ineffective  learner  efforts  (Butler,  1998)  and  may  include  teacher-modeling  of  SR  (Butler,  1998;  Corno  & Randi 
1999;  Maitland,  2000;  Manning,  Glasner,  & Smith,  1996).  Assuming  an  instructor  is  an  integral  and  necessary  part 
of  the  SR  support  imposes  a design  constraint  on  the  learning  context  to  which  the  model  may  be  applied.  Many  of 
the  models  reviewed  for  this  study  depend  upon  relatively  labor-intensive,  skillful  teacher-learner  interactions  to 
provide  individualized  SR  support;  instructors  would  at  a minimum  have  to  be  trained  or  guided  to  fulfill  their  role 
and  even  then  may  not  have  the  time  to  individualize  the  instruction.  On  the  other  hand  their  very  strength  for  a 
specific  purpose  becomes  a weakness  when  seeking  to  embed  instruction  in  different  content/skills,  perhaps  with 
minimum  instructor  effort  to  manage  the  SR  support. 

The  purpose  of  this  developmental  study  was  to  embed  parsimonious  SR  support  requiring  minimal 
preparation  and  maintenance  time  and  effort  and  to  document  that  process;  the  SR  model  applied  to  the  content 
should  facilitate  this  goal.  Therefore  the  model  selected  for  embedding  instructional  SR  support  (Ley  & Young, 
2001)  was  most  appropriate  for  three  reasons.  First,  embedded  SR  features  could  be  used  instead  of  instructor- 
individualized  SR  support;  second,  the  SR  principles  were  readily  adaptable  to  a broad  range  of  content  and 
contexts;  third,  the  model  was  theoretically  and  empirically  supported.  In  sum,  the  model  was  flexible,  data-based, 
and  parsimonious;  it  was  a design  model  well  suited  to  the  investigation.  The  inherent  model  adaptability  has 
distinct  advantages  over  SR  models  constrained  by  learners,  content,  or  learning  context  requirements  or 
assumptions. 


POME  SR  Support  Model 

The  POME  model  encompasses  four  principles  (POME)  that  may  be  flexibly  applied  to  most  instructional 
contexts  and  learning  contents  (Ley  & Young,  2001).  The  principles  establish  four  SR  support  functions 
characterizing  embedded  self-regulation  support  yet  they  are  general  enough  to  be  flexibly  adaptable  to  most  content 
or  contexts.  The  POME  model  must  be  used  with  a content  design  model,  e.g.,  Dick,  Carey,  & Carey  (2001),  or  to 
enhance  existing  instruction.  The  four  SR  principles  establish  instructional  conditions  or  features  in  four  categories: 
preparing  a study  environment,  organizing  study  materials,  monitoring  learning  processes,  and  evaluating  learning 
processes  (POME).  When  applied  collectively  and  systematically  to  create  a comprehensive  SR  support  system,  the 
four  design  principles  guide  embedded  self-regulation  support  design  for  either  existing  or  planned  instruction. 

Principle  1 (Prepare):  Guide  learners  to  prepare  and  structure  an  effective  learning  environment.  Instruction 
should  encourage  and  guide  learners  to  prepare  an  appropriate  learning  environment.  Successful  learners  make 
“efforts  to  determine  or  arrange  where  a task  is  to  be  completed”  (Trawick  & Corno,  1995,  p.  62).  Structuring  the 
environment  relates  to  a learner’s  ability  to  cope  effectively  with  disturbances,  a crucial  part  of  SR  (Como,  1994). 

Principle  2 (Organize):  Organize  instmction  and  activities  to  facilitate  cognitive  and  metacognitive 
processes.  Organizing  is  an  important  component  of  SR  (Hagen  & Weinstein,  1995;  Zimmerman  & Paulsen,  1995). 
Organizing  materials  may  be  broadly  defined  as  transforming  and  “rearranging  instmctional  materials  to  improve 
learning  (Zimmerman  & Martinez-Pons,  1986,  p.  618). 

Principle  3 (Monitor):  Use  instmctional  goals  and  feedback  to  present  the  learner  with  monitoring 
opportunities.  “Monitoring  is  an  important  component  of  self-regulated  learning”  (Zimmerman  & Paulsen,  1995,  p. 
13).  Monitoring  is  the  cognitive  process  that  assesses  the  state  of  progress  relative  to  goals  and  generates  feedback 
that  can  guide  further  action;  it  is  pivotal  in  self-regulated  learning  (Butler  & Winne,  1995). 

Principle  4 (Evaluate):  Provide  learners  with  continuous  evaluation  information  and  occasions  to  self 
evaluate.  Self-evaluation  “involves  the  comparative  outcome  between  some  component  of  performance  and  the  set 
standard”  (Belfiore  & Hornyak,  1998,  p.  190)  or  setting  standards  and  using  them  for  self-judgment  (Zimmerman, 
1998a,  p.  78). 

Embedding  SR  may  avoid  the  constraints  of  instructor-led  SR,  but  would  embedded  SR  prove  practical, 
useful,  effective,  and  sustainable  for  the  instmctor  and  the  students?  The  study  answered  two  research  questions 
about  the  process  of  embedding  systematically-designed  SR  support.  First,  how  easily  can  a designer  apply  a model 
for  embedded  instructional  self-regulation  (SR)  support  into  existing  instruction?  Second,  are  the  resulting 
instructional  features,  that  is,  the  cues,  tools,  prompts,  and  organizers,  effective  and  feasible  to  implement  with 
instructors  and  students? 

Method 

Data  was  collected  during  three  phases  (design,  development,  and  field  test).  A description  of  the  SR 
support  features  precedes  procedure  description  that  details  how  SR  support  in  the  course  was  designed,  integrated 
and  delivered.  Once  the  SR  model  for  the  design  and  the  course  for  the  field  test  was  identified,  the  two  designer- 
researchers  met  to  design  the  SR  support  tools  in  one  session.  The  researcher  who  would  teach  the  class  conducted 
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the  expert-review  interview  to  formatively  evaluate  the  materials  in  an  hour  meeting.  Data  on  SR  support 
implementation  feasibility  and  effectiveness  was  collected  during  the  field  test. 

Participants 

Participants  in  the  design  and  development  phase  included  the  two  experienced,  formally  trained  designers, 
us;  a regular  course  instructor,  an  experienced  teacher  who  evaluated  and  recommended  revisions  to  the  documents; 
the  learners  during  the  field  test.  Both  instructors,  the  regular  course  instructor  and  the  designer-researcher,  had 
similar  supervisor  ratings  on  classroom  observations  (no-SR,  3.  89;  SR,  3.  92  on  a scale  of  1 - 4,  low  to  high) 
although  the  regular  course  instructor  had  more  years  teaching  experience. 

The  learner  participants  were  randomly  assigned  to  alternating  sections  of  the  two  offered  in  the  same 
classroom,  one  at  the  eight  a.  m.  and  the  other  at  nine  a.  m.  The  participants  were  low-achieving  college  students 
who  failed  to  meet  admission  requirements  and  subsequently  assigned  to  this  course.  Over  60  students,  at  least  30  in 
each  section  were  randomly  assigned  to  one  of  two  sections  of  a mandatory  study  skills  course  at  a southern 
community  college.  Fifty-one  of  the  participants  completed  the  course,  21  in  the  no-SR  section  and  29  from  the  SR 
section.  The  course  was  mandatory  for  all  but  one  of  the  participants  because  they  had  an  ACT  score  less  than  17; 
the  remaining  participant  who  was  not  required  to  enroll  in  the  course  and  had  an  ACT  score  of  21  was  dropped 
from  the  analysis.  Participants  in  the  two  sections  had  comparable  ages,  ACT  scores,  and  racial  composition 

Design-Development  of  SR  Instructional  Support 

The  design  assumed  several  conditions  during  the  initial  planning  stages  for  development  of  SR 
instructional  support  given  the  context  of  the  design  problem.  First,  the  target  population,  developmental  or  under- 
prepared college  students,  would  have  SR  deficiencies  associated  with  lower-achieving,  under-prepared  college 
students  (Ley  & Young,  1998).  Second,  the  SR  instruction  would  benefit  all  students,  not  just  low-achievers.  Third, 
the  SR  strategies  had  to  be  largely  intuitive  and,  if  possible,  inherent  to  the  support  structure  or  form  thereby  limiting 
time  spent  on  training  or  learning  to  use  any  SR  features  other  than  required  by  job  aids  or  minimal  instructions. 
Fourth,  the  strategies  would  not  explicitly  teach  SR.  Fifth,  transferring  learner  SR  skills  to  other  learning  contexts 
was  not  a goal;  although  the  use  and  modeling  of  SR  support  throughout  the  course  by  the  instructor  should  motivate 
students  to  deploy  the  strategies  in  other  learning  environments,  it  was  neither  necessary  nor  expected.  Since  the 
four  principles  are  to  be  applied  synergistically,  the  researcher-designers  included  had  at  least  one  support  feature 
within  the  course  for  each  of  the  four  principles.  One  or  more  SR  features,  that  is,  activities  and  tools,  supported 
each  principle  (Appendix  A). 

Preparation  features  direct  students  attention  to  what  and  how  their  study  environment  affects  their 
learning.  Preparing  the  learning  environment  was  presented  with  one  brief,  recurrent  activity.  The  feature  included  a 
six-item  Environmental  Structuring  Checklist  (ESC;  (Appendix  B)  with  three  frequency  response  options,  almost 
always,  sometimes,  almost  never.  Students  were  instructed  to  mark  the  option  that  best  described  how  frequently 
they  studied  under  six  different  statements  represented  four  positive  and  two  distracting  conditions.  For  example, 
students  indicated  frequency  with  which  they  studied  under  that  condition,  “avoid  noise  distractions  such  as 
studying  while  listening  to  television,  music  or  radio”  - almost  always,  sometimes,  almost  never.  The  initial  baseline 
plus  three  exams  were  on  the  first  form  and  last  four  on  a second  form.  The  SR  class  students  recorded  a priori  study 
conditions  for  the  first  examination  and  then  their  subsequent  score  on  the  corresponding  examination;  for  the 
remaining  six,  they  recorded  how  they  studied  on  the  examination  day  and  recorded  the  score  after  the  exam  was 
returned. 

Organizing  in  this  case  is  reinforcing  recognition  of  the  content  to  be  covered  by  a multiple  choice 
recognition  test.  To  add  SR  support  to  existing  instruction  this  SR  feature  presented  a set  of  notes  that  included  the 
salient  information  required  for  the  examinations.  The  Partial  Study  Guide  (PSG;  (Appendix  C)  organized  study 
materials  by  presenting  chapter  content  in  a list  of  statements  with  about  40%  of  key  concepts  missing.  The  student 
completes  the  statements  by  finding  the  corresponding  statement  in  the  text  that  includes  the  missing  content.. 

The  SR  support  design  included  three  different  monitoring  features.  The  first  was  a study  time  log  (STL)  that  listed 
assignments  due  dates,  test  dates  and  included  a monthly  calendar  for  each  month  in  the  semester  on  a different 
page.  The  STL  included  a computer-generated  calendar,  one  month  per  page  with  squares  for  each  day  large  enough 
to  record  a beginning  and  ending  study  time. 

The  second  monitoring  feature,  a Progress  Report  (PR;  (Appendix  D)  listed  all  examinations  and 
assignments  with  a place  to  record  the  completion  date  and  the  number  of  points  earned  or  score.  Learners  were 
responsible  for  recording  their  grades  from  their  returned  examinations  and  activities.  Completing  the  PR  enabled 
each  student  to  monitor  if  and  when  he  or  she  missed  a test  thereby  tracking  their  progress  through  the  course.  At  the 
same  time  the  instructor  inspected  the  STL,  she  also  reviewed  the  PR. 
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The  third  monitoring  feature,  an  attendance  chart,  covered  a standard  size  poster  board  with  a grid  listing 
class  sessions  in  columns  and  student  names  in  rows.  Attendance  for  each  session  was  indicated  by  a different 
colored  marker  or  sticker  affixed  to  the  cell  on  the  grid  for  that  student  and  that  day.  The  different  colors 
corresponding  to  different  attendance  dates  prevented  a student  from  marking  himself  or  herself  present  in  advance. 

Participants  received  a tool  that  was  designed  to  help  them  compare  their  effort  to  their  learning  outcomes, 
that  is,  their  examination  score.  The  Incorrect  Response  Log  (IRL;  Appendix  E)  listed  the  unit  test  number,  the  item 
number  and  two  blank  lines  on  which  the  student  wrote  out  the  correct  answer. 

Content  Instruction 

The  SR  features  had  to  be  designed  to  accommodate  the  existing  content  instructions.  About  90%  of  the 
course  content  was  presented  through  a textbook,  and  the  course  instructor  had  developed  the  instructional  strategy 
that  depended  upon  the  textbook.  The  course  purpose  was  to  help  students  acquire  the  college  success  skills,  e.  g., 
test- taking  strategies,  time  management  and  organizational  skills,  note-taking  techniques,  and  activities  to  heighten 
motivation  and  build  self-esteem.  Developmental  students  received  instruction  in  a face-to-face  study  skills  course. 
The  existing  course  instruction  covered  the  textbook  content  and  the  usual  instructor  who  had  developed  the  course, 
had  been  reinforcing  learner  self-efficacy  intuitively  and  appropriately  during  class  but  did  not  use  any  specific  self- 
regulation support.  The  researcher  who  would  teach  the  treatment  section  of  the  same  course  recorded  how  and  what 
the  usual  instructor  did  and  identified  the  salient  features  of  the  usual  instructor’ s instructional  approach  during  a 
semester  prior  to  the  study.  The  SR  section  researcher-instructor  imitated  the  repetitious  and  easily-replicated 
instructional  style  of  the  usual  instructor.  The  instructor’s  dominant  instructional  strategy  was  to  direct  students  to 
highlight  textbook  (Ellis,  1997)  as  she  reviewed  the  textbook  content  in  chronological  sequence  during  the  class. 

Procedures 

Procedures  for  the  design,  development,  and  field  test  are  reported  in  this  section.  Although  we,  the 
researcher-designers,  were  familiar  with  the  model,  this  was  initial  effort  to  systematically  design  instruction  with 
the  model.  The  article  (Ley  & Young,  2001)  describing  the  model  guided  the  design  process;  this  included  generic 
strategy  descriptions,  such  as,  provide  a checklist  of  environmental  conditions  conducive  to  studying.  Although 
lacking  developed  SR  support  features,  the  article  clearly  presented  principles  and  strategy  descriptions. 

After  selecting  the  model,  we  identified  constraints  on  the  SR  features  to  assure  their  feasible 
implementation.  The  primary  constraint  was  the  amount  of  instructor  time  required  for  implementation  would  have 
to  be  minimal.  On  the  other  hand,  external  monitoring  similar  to  the  form-checking  and  self-recording  procedure 
had  significantly  improved  achievement  among  college  statistics  students  (Lan,  1996).  Therefore  the  design 
included  external  monitoring;  the  instructor  reviewed  selected  SR  documents  briefly  at  scheduled  intervals 
throughout  the  semester.  Within  three  hours  of  beginning  the  initial  design  phase,  a list  of  SR  instructional  support 
features,  their  sequencing  and  integration  within  the  course,  and  the  instructor’s  role  was  completed  and  a design 
document  produced  (Appendix  A).  After  completing  the  design  revisions,  the  researcher  who  would  teach  the  SR 
section  developed  the  five  forms  and  the  attendance  chart. 

The  regular  instructor  participated  as  an  expert  reviewer  during  the  development  phase;  she  was  charged  to 
identify  any  inconsistencies  with  the  course  content  and  assure  the  features  were  feasible  given  the  learners  and  the 
instructional  context.  One  of  us  who  would  teach  the  SR  section  met  with  the  usual  course  instructor  to  solicit 
feedback  about  the  appropriateness  and  feasibility  SR  support  materials.  During  the  meeting,  the  usual  course 
instructor  offered  a few  minor  revisions  but  no  substantive  revisions  to  the  SR  support  tools;  she  quite  unexpectedly 
offered  to  use  the  SR  support  immediately  in  the  forthcoming  semester.  The  researcher  clearly  and  unequivocally 
reminded  the  usual  instructor  that  she  would  be  welcome  to  use  all  the  SR  support  when  study  was  completed,  that 
is,  after  the  next  semester.  The  usual  instructor  said  she  would  comply. 

Each  of  the  two  instructors  taught  one  section  of  the  same  course  taught  on  the  same  days  in  the  same 
room,  back-to-back,  and  in  the  morning.  One  section  was  taught  by  the  instructor  who  had  developed  and  usually 
taught  the  course;  the  other  section  was  taught  by  one  of  us  who  had  studied  the  usual  instructor’s  instructional 
strategy  in  every  for  a semester.  Random  assignment  controlled  for  learner  differences  and  certain  instructional 
conditions  were  replicated  for  the  SR  section  to  control  for  important  instructional  features  unrelated  to  SR. 

To  control  for  teaching  style,  the  researcher  who  taught  the  SR  support  section  of  the  course  replicated  the 
control  instructor’s  instructional  strategy;  the  instructor  reviewed/lectured  on  the  textbook  reading  assignment  and 
told  students  what  to  underline  in  the  textbook.  To  control  for  content,  both  sections  had  the  same  syllabus, 
handouts,  projects,  and  assessment  measures.  The  only  instructional  difference  was  that  the  SR  section  received 
instructional  SR  support. 
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The  SR  section  of  the  semester-long  study  skills  course  was  the  field  test.  The  SR  section  instructor  tracked 
the  SR  field  test  process  by  noting  student  comments,  time  allocated  for  cueing  SR  behavior,  etc.  SR  features 
included  more  than  the  obvious,  print-based  tools;  in  practice  it  also  included  oral  or  print  instructional  guidance  for 
using  the  tools,  and  occasional  instructor  prompts  to  complete  PSGs,  PRs,  STLs  or  ESCs  or  cues  to  use  the  tools, 
such  as  reminders  to  complete  the  PSG  or  allocating  class-time  to  complete  an  IRL. 

The  SR  section  instructor  required  students  to  purchase  a paper  folder  with  center  brads  for  organizing  their 
SR  support  documents  and  instructor  initially  distributed  an  ESC  form  to  each  student  during  the  last  class  meeting 
of  the  first  week  of  the  semester.  After  the  students  completed  the  form  prior  to  the  first  examination,  the  instructor 
explained  the  importance  of  an  appropriate  study  place  and  provided  examples  of  appropriate  study  places,  such  as 
the  library  or  a home  desk  and  to  remove  distractions  such  as  television  or  radio  or  pets.  Participants  returned  the 
ESC  during  the  same  session  so  that  the  instructor  could  keep  the  form.  Then  after  every  examination,  the  instructor 
returned  the  form  to  the  students  after  each  examination  and  asked  them  to  complete  the  section  on  how  they 
prepared  for  the  examination.  Following  the  examination,  the  instructor  asked  them  to  review  their  ESC  form  and 
reminded  them  to  consider  if  their  test  score  reflected  their  study  environment  and  to  consider  if  they  needed  to 
make  changes  in  their  environment  or  should  continue  studying  in  the  current  conditions. 

Prior  to  each  class  session  the  instructor  posted  the  attendance  chart  conveniently  visible  from  anywhere  in 
the  class  so  that  as  they  arrived,  students  recorded  their  attendance  by  initialing  the  cell  for  their  name  and  the 
session  date.  When  the  poster  was  removed,  the  instructor  placed  a black  “A”  for  the  session  for  those  absent  on  that 
particular  day,  thus  preventing  students  from  marking  themselves  attending  a session  at  a later  time.  At  least  once  a 
week  near  the  end  of  class  the  instructor  called  students’  attention  to  the  attendance  chart  and  reminded  them  to  pay 
attention  to  the  number  of  their  absences.  The  instructor  also  reminded  them  that  the  college  attendance  policy 
administratively  withdrawing  students  with  too  many  absences. 

Students  received  the  PR  and  STL  in  the  first  two  weeks  of  the  semester.  The  instructor  told  students  to 
enter  their  grades  on  the  PR  form  and  file  it  in  their  class  folder  so  that  the  instructor  can  review  it  after  each 
examination.  The  instructor  explained  that  the  length  of  time  studying  was  less  important  than  the  quality  of  the 
studying  and  that  learners  who  were  aware  of  how  long  they  studied  did  better  than  those  who  were  not  aware. 
Beginning  on  the  7th  class  day  in  the  third  week,  the  instructor  prompted  students  to  record  when  they  began  and 
finished  studying  for  that  day  on  the  STL. 

The  SR  instructor  told  students  to  keep  the  IRL  in  their  class  folders  and,  during  the  oral  examination 
review,  record  the  correct  answers  to  each  item  they  missed.  The  SR  section  had  the  same  5 multiple-choice 
Scantron-scored  examinations  during  the  semester  as  the  no-SR  control  section.  The  no-SR  control  section  instructor 
returned  the  tests  and  reviewed  each  test,  item-by-item,  identifying  the  correct  response  for  each.  The  SR 
participants  heard  the  same  oral  review  and  completed  an  SR  tool,  an  Incorrect  Response  Log  (IRL).  During  some 
of  the  reviews,  the  SR  instructor  reminded  students  that  the  completed  IRLs  would  serve  as  a study  guide  for  the 
comprehensive  final  examination.  After  returning  the  exams,  the  instructor  encouraged  students  while  the  class  was 
reviewing  answers,  everyone  should  complete  his  or  her  IRL.  The  instructor  did  not  directly  check  the  IRL  form  but 
did  monitor  use  of  the  time  to  complete  the  form.  Students  had  no  substantive  questions  about  how  to  complete  the 
forms  and  expressed  no  difficulty  in  completing  them. 

The  SR  section  instructor  asked  students  to  complete  the  SR  forms  as  part  of  the  class  and,  while  never 
penalizing  or  reprimanding  students  who  did  not  complete  SR  activities,  implicitly  expected  them  to  do  so  by 
checking  every  student’s  class  folder  and  his  or  her  PR,  STL  and  IRL  records. The  instructor  would  check  each 
person’s  SR  activities  as  he/she  submitted  his/her  exam  and  recorded  one  of  three  levels  SR  activity  for  the  set  of 
three  documents,  (1)  progress  reports  (PR),  (2)  study  time  log  (STL),  and  (3)  incorrect  response  log  (IRL);  the 
symbols,  a - minus  if  no  monitoring  for  any  of  the  three;  * check  if  partial  monitoring,  that  is  any  monitoring  on  any 
of  the  three;  + plus  if  completed  monitoring  on  all  three.  The  instructor  reviewed  but  did  not  record  a fourth  SR 
support  document,  (4)  environmental  structuring  checklist  (ESC).  The  purpose  was  to  assure  that  most  of  the  class 
participated  as  expected  and  most  learners  completed  most  of  the  tools  most  of  the  time;  the  review  was  adequate  to 
ensure  that  almost  all  students  engaged  in  the  SR  activities  as  intended.  The  PSG  was  usually  included  in  folders 
although  not  specifically  reviewed. 


Results 

The  initial  purpose  of  the  study  was  to  answer  two  questions,  the  first  of  which  was,  can  a designer 
efficiently  apply  an  SR  support  model  into  existing  instruction?  The  answer  required  inspection  of  design  and 
development  records  kept  by  the  two  designer-researchers  one  of  whom  taught  the  SR  section.  An  analysis  of  the 
documentation  from  each  phase,  design,  development,  and  field  test  process  and  the  observations  of  both  designer- 
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researchers  at  each  phase  of  the  process  answered  the  first  question.  The  second  question  asked  if  the  resulting  SR 
support  would  be  effective. 

To  determine  the  efficiency  of  designing  developing,  and  delivering  the  SR  instructional  support,  the  time 
and  skill  required  were  compared  to  the  quality  of  the  SR  features.  Therefore  efficiency  was  a function  of  time  and 
effort  for  design,  development,  and  delivery  for  sustaining  the  essential  SR  activities,  and  the  quality  of  the  features. 
The  SR  model’s  uncomplicated  and  limited  set  of  explicit  principles  and  the  predetermined  instructional  strategy  for 
delivering  the  content  established  the  SR  requirements  for  the  pre-existing  course.  Designing,  SR  support  for  an 
existing  course  proceeded  quickly  since  the  designers  had  limited  but  explicit  guidelines  to  facilitate  the  process. 

The  designers  had  generic  strategy  descriptions  from  the  initial  article  describing  the  model  that  made  it  relatively 
easy  to  apply  the  model;  the  instructional  strategy  dictated  the  content  of  the  tools.  The  rapid  design  process  yielded 
features  addressing  all  four  SR  support  principles  with  five  specific  tools  (PR,  IRL,  ESC,  PSG,  and  an  attendance 
chart). 

Unsurprisingly,  applying  the  SR  model  required  about  three  hours  each  for  two  designers  to  create  the 
design  for  the  instructional  SR  support  features,  write  brief  instructions  and  format  the  support  features.  Except  for 
the  partial  student  guide  development  of  all  other  components  took  about  2 hours  since  most  were  one  page  forms 
with  minimal  text.  The  partial  study  guide  took  the  most  time  to  develop  since  each  of  seven  was  one  to  three  pages 
long  and  transcribed  from  the  course  content.  The  skill,  and  intellectual  effort  to  deliver  SR  was  minimal,  and  the 
time,  less  than  15  hours  for  the  semester  for  the  cues,  prompts,  and  in-class  external  monitoring  checks. 

The  expert  review  of  the  features  conducted  with  the  usual  course  instructor  by  one  of  the  researcher- 
designers  took  less  than  one  and  a half  hours  including  the  resulting  minor  revisions,  such  as  deleting  words  or 
moving  elements  on  a form.  During  the  review  the  researcher  who  would  be  the  SR  section  instructor,  reminded  the 
usual  instructor,  eager  to  use  the  SR  features  immediately,  that  she  had  to  wait  until  the  following  semester. 

Likewise,  the  development  process  required  very  little  time  and  relatively  low  technical  skills.  The  PR, 

IRL,  and  ESC  took  no  more  than  about  an  hour  each  to  develop  since  they  were  forms  rendered  in  MS  Word. 
Although  the  seven  PSGs  took  almost  no  time  to  design,  developing  all  seven  took  about  as  many  hours,  most  of 
which  was  transcribing  printed  textbook  statements  to  a study  guide  since  the  exam  content  was  predicated  upon 
those  statements  and  both  were  predetermined  by  the  usual  instructor  who  had  previously  developed  the  course 
format  and  content  several  semesters  before.  The  applied  the  model  to  create  an  integrated  set  of  SR  support  tools 
for  an  existing  course. 

The  field  test  supported  relatively  efficient  processes  for  both  the  instructor  and  student.  Their  time 
required  to  sustain  the  SR  activities  was  optimized  because  three  of  the  five  SR  tools,  the  ESC,  PR,  and  STL  were 
inspected  as  students  submitted  their  completed  examinations.  The  combined  review  took  about  30  seconds  to  2 
minutes  per  student  but  no  additional  time  for  the  instructor  because  it  was  completed  while  proctoring  the  class 
during  the  examination.  Students  spent  very  little  time  in  class  on  SR  activities  and  kept  four  of  the  five  SR  tools  in 
a binder,  the  PSG,  IRL,  PR,  and  STL.  Students  completed  the  PSG,  PR,  and  STL  on  their  own  time  and  the  IRL, 
ESC,  and  attendance  chart,  in  class.  Most  students  could  completed  the  IRL  during  the  quiz  oral  review  when  the 
teacher  corrected  the  errors  for  the  class  and  the  amount  of  effort  a student  expended  to  complete  an  IRL  increased 
proportionally  to  the  number  of  missed  items.  The  PSG  required  the  most  effort  of  all  since  every  student  had  to  find 
each  statement  in  the  text  to  complete  it  in  the  PSG.  Students  could  complete  the  ESC,  distributed,  collected  and 
reviewed  on  test  days,  in  less  than  two  minutes  for  each  examination. 

Descriptive  and  inferential  statistics  combined  with  implementation  notes  answered  the  second  question 
about  SR  support  effectiveness.  Two  conditions  violated  basic  assumptions  for  parametric  statistical  analysis:  (a)  the 
student’s  total  points  earned  were  not  distributed  normally  in  the  two  sections  and,  (b)  the  no-SR  section  had  less 
than  30  participants  by  the  end  of  the  semester.  Descriptive  nonparametric  statistics  were  in  the  direction  expected; 
the  SR  section  had  higher  2nd  and  3rd  quartile  scores  (802,  978,  1005  Pts)  than  the  no-SR  section  (890,  927,  956) 
quartile  scores.  Likewise,  the  range  for  the  SR  section  had  a higher  minimum  and  maximum  total  points,  higher 
mean  and  lower  standard  deviation  (380  - 1060;  M = 883;  SD  = 182,  n = 29)  than  did  the  no-SR  section  (0  - 1045; 
M = 848;  SD  = 238,  n = 21).  Despite  mean  scores  were  in  expected  direction  and  higher  relative  range,  the  rank 
orders  between  SR  and  no-SR  sections  were  not  statistically  different  for  a Mann- Whitney  U test;  28.1  for  SR 
support  (n  = 29)  and  21.8  for  no-SR  (n  = 21). 


Discussion 

The  results  of  the  study  reveal  that  systematically  designed  SR  support  is  feasible  to  design,  develop,  and 
implement  efficiently  and  can  result  in  positive  learning  outcomes.  The  POME  principles  applied  to  SR  instructional 
support  design  led  to  an  integrated  set  of  SR  tools  and  activities.  The  embedded  SR  support  model  strongly  guided  a 
very  fast  design  and  uncomplicated  development  process  which  culminated  in  a set  of  embedded  strategic  SR 
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instructional  enhancements.  An  easily-remembered  four-letter  acronym,  POME,  enabled  designers  to  apply  the  four 
principles  without  consulting  complex  charts  or  using  decision  trees.  In  less  than  6 hours  for  design  & formative 
evaluation;  about  1 1 hours  for  development;  and  no  more  than  2 hours  instructor  time  for  the  semester,  SR  support 
was  embedded  in  an  existing  college  study  skills  course. 

Three  features  imbue  the  POME  model  with  useful  advantages  over  many  SR  approaches.  Unlike  many  of 
the  other  approaches,  this  model  promotes  empirically-derived  strategic  SR  activities  without  requiring  any  specific 
media,  or  content  to  implement  the  four  principles.  First,  the  POME  model  may  be  applied  to  most  media  including 
a web  delivery.  Second,  the  instructional  features  suggested  by  the  principles  may  be  delivered  through  web-based 
materials  and  activities  with  minimum  instructor  support.  A third  advantage  specific  to  this  context  was  that 
embedding  instructional  SR  support  based  on  activities  empirically  associated  with  higher  achievement  encouraged 
strategic  learner  self-regulation  and  avoided  labor-intensive  instructor-individualization.  Because  the  foundation  of 
this  model  is  based  upon  investigations  that  addressed  multiple  SR  activities  simultaneously,  the  model  addresses 
the  activities  from  these  multiple-activity  studies  which  are  more  strongly  associated  with  higher  achievement  than 
most  SR  activities.  Therefore  this  is  one  model  that  attempts  to  provide  strategic  support  and  thereby  put  the 
learner’s  instructional  effort  on  activities  which  evidence  suggests  may  contribute  the  most  to  positive  learning. 
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Abstract 

The  purpose  of  this  study  was  to  understand  the  practice  of  online  facilitation  in  a Midwestern  university 
which  has  already  established  a highly  successful  traditional  MBA  program.  This  article  explored  the  instructors  ’ 
perceptions  regarding  four  dimensions  of  instructor  roles  using  Berge’s  (1995)  classifications:  pedagogical, 
managerial,  social,  and  technical.  This  article  also  examined  the  challenges  and  issues  confronting  online 
instructors  when  fulfilling  these  roles.  The  results  suggest  that  instructors  carried  out  several  important  roles  to 
varying  degrees.  The  findings  reveal  a stronger  emphasis  on  the  pedagogical  roles  (e.g.  course  designer,  profession 
inspirer,  feedback  giver,  and  interaction  facilitator).  A lesser  emphasis  on  social  roles  represented  mixed  feelings 
regarding  its  importance  to  the  instructors.  While  students  rated  the  instructors  very  positively,  the  results  also 
indicate  that  instructors  still  need  to  have  their  roles  transformed  pedagogically,  socially,  and  technologically  if 
they  are  to  establish  a more  engaging  and  fruitful  environment  for  online  learning. 

Introduction 

One  of  the  key  challenges  facing  online  instructors  is  how  to  provide  clear  and  visible  guidance  in  a virtual 
environment.  In  the  traditional  classroom,  facilitation  relies  on  both  verbal  and  non-verbal  cues  to  initiate 
understanding  of  ongoing  communication  and  courses  tasks.  Communication  online,  however,  relies  mainly  on 
written  language  without  paralinguistic  cues.  Changes  in  communication  patterns  require  instructors  to  adapt 
personal  perceptions  of  their  roles  in  order  to  adjust  to  an  online  learning  environment  that  keeps  remotely 
distributed  learners  continuously  engaged  in  the  learning  process  (Copplola,  Hiltz,  & Rotter,  2001).  A large  body  of 
literature  argues  that  the  roles  of  online  instructors  may  be  new  and  more  complicated  than  in  traditional  classrooms 
(Bonk,  2000;  Heuer  & King,  2004).  Meanwhile,  the  emergence  of  new  technologies  makes  it  increasingly  easier  to 
have  distant  learners  involved  in  two-way  communications,  thereby  enabling  self-directed  learners  to  construct 
meaning  both  individually  and  socially  (Berge,  2001).  The  culture  of  higher  education  has  increasingly  moved 
away  from  teacher-centered  learning  towards  learner-directed  learning,  with  instructors  functioning  as  facilitators, 
coaches,  consultants,  or  resource  people  who  provide  intellectual  guidance  in  student  learning  (Berge,  2001;  Bonk, 
wisher,  & Lee,  2003). 

A comprehensive  classification,  first  proposed  by  Berge,  then  expanded  by  Ashton,  Roberts,  and  Teles 
(1999),  describes  the  multiple  roles  of  online  instructors  under  four  dimensions:  (1)  pedagogical,  (2)  social,  (3) 
managerial,  and  (4)  technical.  While  there  is  a growing  body  of  literature  that  emphasizes  the  importance  of 
instructor  roles  in  distance  learning,  few  empirical  studies  have  examined  the  issues  that  instructors  face  in  their  new 
roles.  Ashton  et  al.  suggest  future  research  should  examine  the  four  dimensions  of  online  instructor  roles  across 
different  instructors,  across  different  courses,  and  from  the  beginning  to  end  of  an  online  course.  The  results 
reported  in  this  study  extend  previous  findings  regarding  the  facilitative  roles  of  online  instructors  by  examining 
pedagogical,  social,  managerial,  and  technical  roles  across  instructors  in  an  online  program. 

The  purpose  of  this  research  was  to  examine  the  most  important  concerns  and  issues  linked  to  the  four 
dimensions  of  online  instructor  roles  in  a rapidly-expanding  online  MBA  program.  This  study  will  focus  on  the 
following  research  questions: 

• What  are  instructors’  perceptions  of  their  roles  when  teaching  online  in  terms  of  Berge’s  classifications? 

• What  are  students’  perceptions  of  four  dimensions  of  instructor  roles? 

Theoretical  Review 

In  the  present  research,  Berge’s  instructor  roles  - pedagogical,  managerial,  social,  and  technical  - have  been 
used  as  a starting  point  to  help  organize  the  literature  on  the  varied  duties  of  the  online  teacher  as  well  as  the  key 
findings  of  this  study. 
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The  pedagogical  roles  of  the  online  instructor  revolve  around  facilitating  educational  processes  for 
students’  understanding  of  critical  concepts,  principles,  and  skills  (Berge,  1995).  Such  tasks  include  encouraging 
students’  knowledge  sharing  and  building  through  interactive  discussion,  designing  a variety  of  educational 
experiences,  providing  feedback,  and  referring  to  external  resources  or  experts  in  the  field.  Similarly,  Salmon  (2000) 
elaborated  the  important  role  of  facilitators  as  “weavers”  who  pull  together  the  participants’  contributions  in  an 
online  conference  by  weaving  statements  and  relating  them  to  relevant  concepts  and  theories  so  as  to  enable  the 
development  of  ideas  through  collaborative  discourse.  High  quality  instructions  require  online  moderation  skills  of 
instructors.  Oftentimes  a variety  of  pedagogical  roles  have  to  be  performed  to  facilitate  higher  order  thinking  skills. 

The  social  function  is  typically  employed  to  promote  a friendly  environment  and  community  feel  that 
supports  student  cognitive  learning  processes.  Such  social  tasks  include  developing  harmonic  human  relations, 
group  cohesiveness,  and  collective  identity  (Berge,  1995).  Online  social  roles  require  the  instructor  to  develop 
“nurturing”  skills  for  creating  a context  conducive  to  knowledge  construction  by  encouraging  participation,  giving 
ample  feedback  and  reward,  attending  to  individual  concerns,  and  using  a friendly,  personal  tone  (Kerr,  1986).  It  is 
expected  that  in  a learning  community,  participants  have  a strong  sense  of  belonging  and  are  ready  and  willing  to 
contribute  to  the  knowledge  building  of  a larger  community  (Wenger,  1991).  Though  the  social  roles  have  been 
widely  cited  in  literature,  studies  have  indicated  that  instructors  who  are  new  to  teaching  online  may  have  difficulty 
getting  used  to  this  concept.  Conrad ‘s  (2004)  study  noted  that  the  inexperienced  online  instructors  often  lacked 
essential  social  skills  and  preferred  personalized  communication  instead  of  taking  leading  roles  in  establishing  a 
wider  community. 

The  managerial  roles  include  the  organizational,  procedural,  and  administrative  tasks  associated  with  the 
learning  environment  (Berge,  1995).  The  tasks  involve  coordinating  assignments,  managing  online  discussion 
forums,  and  handling  overall  course  structure  (Ashton  et  al.,  1999).  Berge  advocates  that  online  instructors  take 
strong  leadership  in  shaping  online  interaction  through  setting  clear  agendas  and  objectives  of  online  conferences 
and  establishing  procedural  rules  and  decision-making  norms.  Studies  support  using  the  course  syllabi  and  time 
tables  to  lay  out  the  expectations  of  learning  activities  to  improve  the  performance  of  online  discourse  (Pincas, 

1998) , and  setting  the  expectations  for  the  conditions  of  communication  prior  to  the  beginning  of  the  course  to 
shape  the  nature  and  the  amount  of  interaction  during  the  course  (Heuer  & King,  2004).  The  evolving  nature  of  the 
managerial  roles  has  been  observed  in  several  studies  (Concrad,  2004).  The  managerial  roles  may  not  be  directly 
related  to  learning,  but  they  are  regarded  as  important  roles  as  they  constitute  necessary  conditions  for  maintaining  a 
successful  online  learning  environment. 

Early  studies  indicated  that  technical  difficulty  was  one  of  the  most  important  reasons  that  caused  students’ 
dissatisfactions  with  web-based  courses  (Hara  & Kling,  2000).  The  technical  roles  of  online  instructors  can  make 
participants  comfortable  with  the  system  and  software  program  used  for  online  courses.  Such  tasks  include  referring 
students  to  technical  support  resources,  addressing  technical  concerns,  diagnosing  and  clarifying  problems 
encountered,  and  allowing  students  sufficient  time  to  learn  new  programs  (Ashton  et  al.,  1999).  By  accomplishing 
the  technical  roles  successfully,  the  learners  will  be  able  to  concentrate  on  the  academic  task  (Berge,  1995). 

Methodology 

The  purpose  of  this  study  was  to  understand  the  nature  of  instructor  roles  and  the  complex  issues  involved 
in  a distance  learning  environment.  The  case  study  approach  is  considered  appropriate  for  such  exploratory  research. 
Such  an  approach  is  also  beneficial  in  providing  better  understanding  of  a complex  system  like  the  one  studied  here 
(Stake,  1994). 

The  field  setting  selected  was  an  accredited  online  MBA  program  at  a top  ranked  business  school  in  a large 
Midwestern  university.  The  program  was  designed  for  professionals  who  wish  to  continue  their  employment  while 
earning  their  MBA.  The  researchers  believed  that  such  a rapidly  expanding  program  would  best  illustrate  the 
complexities  and  changing  nature  of  instructor  roles. 

For  the  purposes  of  this  particular  study,  twenty-eight  faculty  members  within  this  program  were 
interviewed.  Importantly,  the  subject  areas  of  the  participants  related  to  all  the  major  disciplines  offered  in  this 
program.  Twenty- seven  semi- structured  questions  were  asked  and  the  interview  was  conducted  in  a one-on-one 
interview  format.  The  questions  asked  related  to  the  four  dimensions  of  instructor  roles,  including  issues  of  course 
delivery  and  management,  pedagogy  and  motivation,  social  interaction,  and  technical  support.  Each  interview  took 
approximately  45-75  minutes. 

Strauss  and  Dorbin’s  constant  comparative  method  (Meriam,  1998)  was  used  to  triangulate  the  data  from 
different  interview  transcripts  and  to  identify  emerging  themes  under  different  dimensions  of  instructor  roles.  To 
ensure  the  intercoder  reliability,  two  interview  transcripts  were  coded  initially  and  the  inter-coder  agreement 
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achieved  89  percent.  The  researchers  then  continued  coding  independently  and  discussed  their  coding  decisions  with 
each  other  until  a common  set  of  codes  based  on  all  the  transcripts  were  determined. 

Program  evaluation  survey  data  related  to  students’  perceptions  of  the  learning  online  experience  was  used 
to  assess  students’  satisfaction  with  the  four  dimensions  of  online  instructors’  roles  mentioned  previously.  The 
student  survey  instrument  consisted  of  65  questions.  The  65  item  survey  questionnaire  contained  5-point  scale  Likert 
type  questions  about  their  overall  student  perceptions  and  attitudes  toward  learning  online.  The  internal  reliability  of 
the  survey,  Cronbach’s  alpha,  was  reported  .91. 

Instructors’  perceptions  on  four  dimensions  of  their  roles 

In  this  section,  instructors’  perceptions  of  their  roles  are  described  in  terms  of  Berge’s  four  dimensions. 
Under  each  dimension,  the  roles  were  identified  by  grouping  similar  themes  associated  with  that  title. 

Pedagogical  role 

Course  designer.  Many  instructors  agreed  on  their  “designer”  role  of  repurposing  learning  materials  from 
the  traditional  classroom  to  online  courses.  Due  to  the  loss  of  real-time  interaction  and  rich  contextual  cues  that  they 
had  experienced  in  the  traditional  classroom,  it  was  essential  for  instructors  to  provide  much  more  elaborate 
information  in  their  online  courses.  As  an  example  of  this  change,  one  instructor  shared  the  following: 

...  I’ll  spend  five  minutes  talking  about  an  experience  that  I had,  it’s  more  difficult  to  do  that  with  our  text 
files,  and  that’s  one  of  the  reasons  that  I started  doing  these  sound  clips,  was  to  simulate  some  of  things  that  I 
might  say  in  class  in  addition  to,  or  that  would  supplement,  the  slide  material.  So,  enriching  the  material,  the 
presentation,  as  much  as  possible,  I think  is  the  most  important  point. 

While  selection  and  design  of  course  material  were  crucial,  many  instructors  noted  that  it  was  equally 
important  to  carefully  structure  and  organize  the  course  materials  in  a way  that  made  students  engage  in  learning 
through  a variety  of  activities. 

Many  instructors  also  alluded  to  the  iterative  nature  or  “trial  and  error”  of  online  course  design  by 
continuously  refining  and  improving  the  assignments  or  deliverables  or  course  materials  in  a way  that  was  both 
challenging  and  manageable  from  instructor’s  perspective. 

Instructor  design  strategies  seemed  to  benefit  a great  deal  from  sharing  and  discussing  their  experiences 
with  instructors  who  taught  the  same  courses.  Instructors  worked  as  a team  formally  or  informally  to  discuss  ideas, 
share  materials,  and  mentor  each  other.  Additionally,  they  often  provided  vital  support  and  encouragement  for  the 
faculty  who  were  new  to  teaching  online.  The  interactions  of  instructors  across  different  disciplines  were  not  as  high 
as  within  each  domain  community. 

Faculty  members  in  this  online  MBA  program  seemed  to  have  identified  strongly  with  such  varying  roles. 
This  study  indicated  these  online  instructors  regarded  course  design  as  a vital  first  step  to  ensure  the  quality  of 
online  learning. 

Profession  insyirer.  Helping  learners  move  closer  to  their  professional  aspirations  is  a key  goal  of 
professional  education.  Instructors  consistently  noted  that  the  nature  of  online  learners,  usually  plugged  into  their 
careers  and  more  practically-oriented  than  traditional  learners,  created  a further  impetus  for  them  to  structure  the 
learning  tasks  around  real  world  experiences.  A number  of  instructors  noted  that  they  adjusted  their  assignments  and 
discussions  to  connect  to  students’  work  experience.  One  instructor  commented  on  her  experiences  to  initiate  a 
professional  dialogue  among  online  learners: 

I’m  just  a facilitator  to  draw  out  the  experiences  ....  So  I really  spend  a lot  of  time  trying  to  draw  from  them 
their  experiences  and  then  having  other  people,  you  know,  pick  up  on  what’s  said  and  expand  that  to  their 
environment  and  all  these  things. 

This  opportunity  for  applying  learning  to  the  real  world  is  even  more  handy  when  a number  of  instructors 
taught  company  specific  courses  such  as  a Strategic  Marketing  Management  course  designed  specifically  for 
General  Motors  Corporation.  They  noted  that  online  courses  have  the  advantage  to  take  learners  back  to  their  work 
contexts  and  apply  the  theories  on  the  specific  issues  they  confronted  in  a company. 

Feedback  giver  Not  surprisingly,  instructors  unanimously  agreed  on  the  important  role  of  giving  feedback 
to  online  learners.  The  immediacy  and  quality  of  feedback  as  well  as  the  sustainability  of  instructional  feedback  on 
students’  engagement  was  valued  by  online  instructors.  Two  instructors  commented: 
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Deliverables,  timely  feedback,  and  extensive  feedback  and  forcing  them  to  do  something  with  that  feedback 
[are  essential].  If  I’m  going  to  go  to  the  trouble  to  spend  2 hours  to  write  them  a memo  of  feedback  I want 
them  to  have  to  take  that  feedback  and  use  that  feedback  rather  than  spend  5 minutes  reading  over. 

The  importance  of  feedback  giver  role  reflected,  partly,  the  instructors’  perception  of  the  importance  of 
student-instructor  interaction  and  the  need  for  instructors  to  contribute  their  knowledge  through  ways  of  giving 
feedback.  On  the  other  hand,  instructors  also  used  it  as  the  most  important  way  to  pace  or  control  the  learning 
progress  of  online  learners.  For  example,  an  instructor  commented  on  how  he  redesigned  the  learning  tasks  so  that 
more  interim  feedback  can  be  obtained  from  the  instructors: 

We  use  fewer  cases  but  in  greater  depth  over  a longer  period  of  time  with  interim  submissions  by  the 
students  with  interim  feedback  from  the  faculty.  And  all  of  that  is  our  attempt  to  compensate  for  the  lack  of 
real  time  class  discussion  so,  one  way  to  look  at  it  is,  we  do  fewer  assignments  but  another  way  to  look  at  it 
is  we  so  amplify  the  ones  that  there’s  more  grading  and  evaluation. 

Interaction  Facilitation.  Instructors  agreed  that  online  discussion  was  the  key  to  online  interaction.  The 
online  discourse  promoted  a democratic  environment  as  it  created  a common  framework  for  students  to  share  their 
learning  experiences  and  professional  growth.  The  instructors  agreed  that  the  students  needed  to  take  a central  role 
in  online  discussion  while  they  assumed  a “facilitator”  or  “consultant”  role  to  scaffold  the  discussion  on  the  road. 

For  example,  one  instructor  commented, 

So  that  I could  find  a way  where  I feel  comfortable  commenting  in  the  discussion  forums  as  another  person 
but  maybe  I have  to  bring  a little  more  expertise  or  a broader  view  of  how  your  company  relates  to  other 
companies.  And  occasionally  students  would  clearly  put  me  in  that  role.  They’d  say  you  know  I had  this 
question  about  your  case.  I wonder  what  the  professor  thinks  about  that. 

Although  many  agreed  on  the  role  of  instructor,  the  range  of  facilitations  showed  a wide  spectrum  of 
moderating  strategies  and  frequencies  of  interactions.  On  the  low  end,  approximately  half  of  the  instructors  rarely 
moderated  online  discussions.  On  the  high  end,  a few  instructors  participated  extensively  in  online  discourse  through 
a variety  of  facilitating  techniques  including  questioning,  prompting  responses,  recognizing,  requesting  responses, 
and  modeling  social  presence. 

Instructors  noted  several  issues  that  affected  the  degree  of  their  presence.  For  instance,  a few  instructors 
expressed  concerns  about  not  knowing  effective  moderating  strategies  to  promote  peer  interaction.  A few  instructors 
were  concerned  about  the  time  commitment  associated  with  facilitating  online  discussion.  In  addition,  some 
instructors  were  concerned  that  the  authoritarian  role  of  the  online  instructor  might  deter  a fruitful  peer  discussion 
among  online  learners. 

Managerial  role 

Conference  manager.  Ensuring  “equity”  in  online  discussion  has  been  one  of  the  most  important 
managerial  concerns.  Many  instructors  noted  that  the  negative  effect  of  “time  lag”  in  asynchronous  discussion  could 
easily  cause  “dominancy”  issues  in  discussions  because  early  students  can  take  control  of  the  forum  by  effusive 
comments.  The  following  comment  demonstrated  how  an  instructor  provided  guidelines  to  ensure  students’  equal 
opportunities  in  contributing  to  the  discussion: 

So  this  year,  I said  you  can  only  contribute  on  the  first  day  to  two  of  the  cases.  You  can’t  contribute  to  all 
four.  Because  there  were  guys  in  the  previous  years  that  would  try  to  beat  everybody  to  the  punch  on  the 
cases,  and  answer  every  question  on  every  case,  and  it  got  some  of  the  others  upset,  so  I said,  no,  you  can 
only  post  to  two.  And,  I give  them  some  suggestions  on  these  things.  You  don’t  have  to  answer  every 
question.  I’m  interested  in  quality,  not  quantity.  This  time,  they  were  much  more  disciplined. 

Another  issue  related  to  “time  lag”  was  the  reduced  responsiveness  of  online  students  to  peer’s  comments 
which  resulted  in  extensive  redundant  information.  To  address  this  issue,  some  online  instructors  created  specific 
rules  related  to  not  repeating  previous  comments  or  ideas.  Besides  introducing  rules  of  non-repeating,  the  instructors 
either  introduced  more  discussion  threads  at  the  beginning  or  introduced  new  discussion  topics  in  the  middle  of  the 
discussion  to  augment  the  discussion. 

Though  much  effort  was  made  on  promote  interactive  discussion,  the  results  revealed  that  the  instructors 
still  had  great  difficulty  in  refining  and  “weaving”  discussion  so  that  conversations  progressed  from  sharing 
answers  to  knowledge  negotiation  and  construction  (Anderson  et  al.,  2001). 
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Organizer  and  planner.  One  important  aspect  of  organizer  was  to  require  instructors  to  be  clearer  and  more 
organized  when  giving  directions  online.  Many  instructors  found  that  any  ambiguity  may  result  in  misunderstanding 
in  communication  and  thus  reduce  the  efficiency  of  the  learning  process.  An  instructor  commented: 

I think  you  have  to  be  clearer  online.  If  you’re  not  clear  in  the  classroom  a student  is  going  to  ask  you  a 
question  and  it’ll  get  conveyed  immediately  to  everybody  ...  So  you’ve  got  to  be  a lot  more  careful  online 
as  to  how  you  answer  something  or  ask  something  and  you’ve  got  to  be  a lot  clearer  in  your  direction, 
clearer  in  your  instruction. 

In  addition  to  daily  pedagogical  concerns,  instructors  consistently  noted  the  importance  of  course  structure 
and  organization  in  an  online  course  for  planning  purposes.  The  instructors  sensed  the  needs  of  online  learners  to  be 
provided  with  a clear  structure  and  timeline  to  keep  them  plugged  in  learning  in  the  midst  of  their  busy  work 
schedules.  For  example,  one  instructor  commented: 

In  an  online  environment  they’ve  got  control  of  their  calendar  essentially  as  to  when  they  do  certain  things. 
The  splitting  up  into  weekly  modules  in  part  was  a recognition  that  we  all  tend  to  put  things  off  if  we  can 
and  so  having  weekly  modules  with  some  sort  of  a deliverable  every  week  . . . puts  or  forces  some  structure 
on  them  so  they  can’t  wait  until  the  last  couple  of  weeks  and  try  to  do  everything  at  that  point  in  time. 

Meanwhile,  flexibility  should  be  considered  by  instructors  to  accommodate  the  students’  schedules.  A 
number  of  instructors  commented  on  their  changes  of  the  assignment  due  date  from  Sunday  to  Monday  which 
seemed  to  work  much  better  because  many  students  have  to  spend  a great  deal  of  weekend  time  to  work  on  the 
assignments. 

Social  role 

Social  rapport  builder.  One  of  the  greatest  challenges  for  online  learning,  as  many  instructors  noted,  was 
the  ability  to  build  a personal  touch  between  the  instructors  and  students.  The  “impersonal  nature”  of  online 
environment  posed  a great  difficulty  for  building  social  rapport.  Examples  of  the  lack  of  personalization  can  be  seen 
in  the  comment  as  following: 

The  feedback  that  we’ve  gotten  from  the  students  is  that  things  online  feel  very  impersonal  and  they  never 
get  to  know  the  instructor.  And  I would  agree  because  I never  get  to  know  the  students. 

Many  instructors  associated  the  low  social  presence  in  their  courses  with  the  limitations  of  existing 
technology  that  was  unable  to  project  participants’  identity  or  personality  into  an  online  course.  To  address  this 
issue,  the  program  established  a policy  to  add  at  least  one  video  introduction  of  the  instructor  in  each  course.  This,  in 
part,  solved  the  problems  of  instructor  identity  in  an  online  course  but,  nevertheless,  did  not  help  with  the  social 
presence  or  online  identities  of  students.  In  fact,  only  two  out  of  27  instructors,  tried  to  use  some  simple  social  ice- 
breaking activities  or  introduction  forums,  which  allowed  students  to  become  familiar  with  each  other  and  their 
personal  stories.  In  addition,  only  3 of  them  used  virtual  office  hours. 

With  regard  to  instructor’s  role  in  building  a sense  of  community  in  an  online  course,  many  admitted  that 
they  did  not  feel  a sense  of  community  in  their  course  nor  did  they  take  any  measures  to  purposively  build  a 
collective  identity  in  an  online  course.  For  instance,  one  instructor  stated: 

Sol  really  encouraged  the  students  to  contact  me  in  any  way  that  they  felt  comfortable.  I sort  of  gave  them 
many  options.  But  as  far  as  creating  a sense  of  we’re  all  here  at  this  moment,  I did  not. 

However,  instead  of  taking  an  active  role  in  establishing  an  online  class  community,  the  instructors  agreed 
on  their  role  in  establishing  online  teams  and  encouraging  group  knowledge  sharing  to  accomplish  a shared  learning 
goal.  Nonetheless,  whether  the  relationship  among  a group  can  be  projected  as  a sense  of  belonging  to  a class 
community  remained  an  open  question.  When  asked  whether  there  was  a sense  of  community  in  his  course,  an 
instructor  indicated: 

Well,  I think  it's  important . . .The  teams  provide  an  opportunity  to  do  that,  whether  or  not  it  has  to  move  to 
the  class  level,  I'm  not  so  sure.  It  probably  is  useful,  but  I'm  just  trying  to  think  about  why  that  would  help 
beyond. 

Several  instructors  mentioned  that  the  time  expense  associated  with  playing  the  social  role  in  an  online 
course  might  be  detrimental  to  their  pedagogical  role  play. 

In  summary,  the  instructors  had  mixed  feelings  regarding  the  importance  of  the  social  role  in  this  online 
MBA  program.  In  general,  these  instructors  were  not  yet  convinced  of  the  relevance  and  viability  of  the  social  role 
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for  student  learning.  Various  technological  limitations  and  concerns  about  time  further  detracted  from  any  efforts  in 
building  a sense  of  community. 

Technical  role 

Technical  coordinator.  Instructors  recognized  that  their  technical  role  required  them  to  refer  students  to 
technical  support  resources  or  communicating  technical  issues  to  support  staff.  This  role  was  also  partly  defined  by 
the  technical  support  structure  of  the  program.  Each  course  was  supported  by  a course  designer  who  helped  with  the 
operation  of  the  course.  Because  the  course  management  system  in  general  functioned  effectively,  sometimes  the 
issue  was  not  really  technical  but  the  inappropriate  creation  of  the  settings  by  the  instructors  or  students’  insufficient 
understanding  of  the  mechanics  of  the  tools.  Oftentimes,  the  commercial  simulation  programs  used  in  the  online 
MBA  courses  had  technical  issues  that  the  instructors  had  to  send  or  forward  to  technical  support  staff. 

Media  designer/ dev  eloper.  About  half  of  the  instructors  reported  uses  of  multi-media  elements  in  their 
courses.  However,  the  instructors  reacted  differently  about  their  roles  in  using  different  forms  of  multimedia  in  their 
online  courses.  Several  instructors  who  had  experiences  in  using  video  tapes  in  the  traditional  classroom  felt 
comfortable  in  converting  their  video  into  digital  format.  However,  a few  instructors  who  had  developed  video  clips 
from  scratch  were  unsure  about  whether  the  anticipated  education  benefit  was  worth  the  extra  development  effort. 

A number  of  instructors  mentioned  the  needs  to  identify  or  develop  tools  that  could  improve  the  efficiency  of 
assessment  and  giving  feedback  to  online  students.  A few  instructors  in  this  program  had  collaborated  with  the 
technical  developers  on  developing  innovative  teaching  tools  through  contributing  their  design  ideas. 

Technology  integrator 

One  of  the  most  highly  mentioned  issues  related  to  the  instructors’  frustration  with  the  inefficiency  of 
online  chat  room  tools.  From  the  start,  almost  every  instructor  tried  to  use  chat  rooms  for  online  discussions  or 
virtual  office  hours.  However,  a majority  of  them  finally  discontinued  using  it  as  a discussion  tool  for  learning. 
Among  many  reasons,  the  concerns  about  technological  limitations,  typing  skills,  and  the  difficulty  in  scheduling  a 
time  for  class  to  meet  online  were  expressed  frequently. 

Many  instructors  held  positive  attitudes  toward  using  the  emerging  collaborative  technologies  such  as  web- 
based  video  conferencing  and  NetMeeting,  which  could  overcome  the  weakness  of  text  chat  tools  and  further 
improve  the  efficiency  and  effectiveness  of  online  learning.  With  those  tools,  it  was  expected  that  it  could  help 
establish  a better  professional  intimacy,  realism,  and  real-world  flavor  for  online  students.  However,  when  adopting 
more  visually  rich  and  interactive  tools,  accessibility  and  bandwidth  were  two  key  issues  or  concerns  of  all  the 
instructors.  Examples  of  those  issues  can  be  seen  in  the  following  comment: 

I think  the  idea  of  doing,  there’s  a lot  you  could  do  but  the  constraint  is  the  computers  at  the  other  end  are  not 
what  they  need  to  be.  I’d  love  to  get  on  a web  cam  and  have  a chat  room  with  a web  cam  where  they  could 
get  on  and  see  me  but  most  of  them  can’t  do  that  because  they  don’t  have  the  software  or  the  hardware  to  do 
it. 

The  role  of  “technology  integrator”  of  online  instructors  represented  an  increased  awareness  and  effort  of 
highly  interactive  pedagogical  tools.  However,  the  issues  involved  with  technology  integration  affected  the 
technological  roles  of  online  instructors.  As  a result,  the  overall  level  of  technology  use  was  still  relatively  low. 
Email,  asynchronous  discussion,  and  announcement  online  were  the  most  frequently  employed  communication 
tools,  whereas  more  sophisticated  interactive  tools,  e.g.  online  synchronous  collaborative  tools,  multimedia  lectures, 
etc.,  were  in  relatively  low  use. 

Students’  perceptions  of  four  dimensions  of  instructor  roles 

The  selected  items  from  program  evaluation  survey  reflected  students’  satisfactions  of  four  dimensions  of 
instructor  roles. 

Pedagogical  role 

According  to  the  student  survey,  overall,  the  students  had  positive  experiences  with  instructor’ s 
pedagogical  facilitation  in  this  program,  and  were  highly  satisfied  with  the  impact  of  instructor  guidance  on  their 
learning  experiences.  Nearly  80  percent  of  the  students  agreed  that  the  instructors  used  a variety  of  instructional 
techniques  to  foster  students’  critical  and  reflective  thinking.  About  90  percent  of  the  students  agreed  that  online 
learning  activities  fostered  their  understanding  of  key  concepts.  Nearly  85  percent  of  the  students  agreed  that  the 
instructors  play  a role  of  “facilitator”  rather  then  a “lecturer.”  More  than  75  percent  of  the  students  replied  that  the 
way  the  instructors  facilitated  the  class  discussion  fostered  their  learning.  However,  about  20  percent  of  them  were 
not  so  sure  about  the  effectiveness  of  the  instructors’  facilitation  in  discussion  forums  . 
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Social  role 

The  results  suggested  that  the  level  of  social  presence  in  this  program  was  relatively  low.  In  effect,  the 
students  were  not  engaged  in  a fully  socially  supportive  online  learning  environment.  While  approximately  60 
percent  of  the  students  never  felt  lonely  while  taking  the  classes,  about  25  percent  did,  in  fact,  feel  isolated.  In 
addition,  49  percent  could  not  saliently  feel  the  emotions  or  feelings  of  other  students  in  their  online  courses. 
Similarly,  only  28  percent  students  felt  that  they  knew  their  instructors  and  other  students  well.  Although  the 
students  felt  a low  level  of  social  presence  in  online  courses,  nearly  90  percent  indicated  they  felt  they  were  part  of  a 
learning  community  when  they  took  online  courses.  These  findings  indicated  that  other  factors,  such  as  the  one  week 
in-residence  required  by  the  program  to  develop  group  cohesiveness  helped  foster  a sense  of  community  in  online 
courses  in  this  program. 

Managerial  role 

The  students  overall,  reacted  positively  about  the  organization  of  course  materials  and  were  able  to 
efficiently  find  learning  resources  in  an  online  course.  About  85  percent  of  online  students  agreed  that  online 
instructors  provided  clear  instructions.  Nearly  95  percent  of  the  students  agreed  that  the  online  courses  were  well 
organized.  With  the  effort  of  online  instructors  in  promoting  equity  in  online  participation,  nearly  nine  in  ten 
students  felt  comfortable  participating  in  online  discussion  with  others. 

Technical  role 

The  students  overall  had  positive  experiences  with  technical  tools  used  in  online  courses  and  rated  highly 
the  overall  program  technical  support.  Slightly  more  than  85  percent  of  the  students  agreed  that  the  tools  and 
technologies  used  in  online  courses  were  helpful  for  deep  learning.  However,  it  should  be  noted  that  about  11 
percent  of  the  students  did  not  agree  that  technologies  were  effectively  used  in  online  courses. 

The  overall  satisfaction  of  learners  in  this  program  was  extremely  high.  Students  in  this  study  had 
significantly  benefited  from  taking  courses  in  this  online  MBA  program  and  were  able  to  utilize  the  knowledge 
learned  from  online  courses. 


Discussion 

Although  the  themes  presented  may  not  represent  all  the  roles  the  instructors  engaged  in,  they  represented 
the  most  important  concerns  expressed  by  online  instructors  in  this  program.  The  findings  confirmed  Berge’s 
assertions  that  instructors  perform  different  roles  in  different  degrees  when  teaching  online.  Many  roles  identified  in 
this  paper  are  similar  to  what  have  been  identified  in  previous  research.  At  the  same  time,  there  are  some  emerging 
new  roles,  such  as  “profession  inspirer”,  that  have  not  been  discussed  in  literature.  Based  on  the  findings  from  this 
study,  a summary  of  the  perceived  priority  of  different  roles  by  online  instructors  is  displayed  in  Figure  1 (3=High 
priority,  2=Medium  priority,  l=Low  priority). 


□ Priority  level 


Figure  1.  Perceived  priority  of  different  roles  (High  priority=3,  Medium  priority=2,  Low  priority=l) 
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Overall,  there  is  a much  stronger  emphasis  on  the  pedagogical  roles  within  four  dimensions.  The 
effectiveness  of  instructor’s  pedagogical  roles  was  confirmed  by  the  students’  survey.  The  strong  emphasis  on  the 
pedagogical  roles  reflected  four  important  concerns  of  the  pedagogical  roles:  (1)  course  design,  (2)  promoting 
professional  aspirations,  (3)  providing  timely  and  quality  feedback  and  (4)  facilitating  discussion.  By  emphasizing 
those  tasks,  the  instructors  promote  three  key  types  of  interactions  online:  student-content,  student-student,  and 
student-teacher. 

It  is  worth  noting  that  a strong  emphasis  on  the  “designer  role”  reflects  an  important  a policy  concern  in 
this  online  MBA  program;  namely,  the  importance  of  ensuring  that  the  learning  experience  of  students  in  the  online 
environment  is  roughly  equivalent  to  what  students  experience  in  the  traditional  classroom  environment.  Based  on 
the  results  of  this  study,  it  is  clear  that  traditional  course  materials  provide  a solid  knowledge  foundation  for  online 
courses  to  build  on  (Salmon  & Giles,  1997).  However,  it  was  also  observed  that  mimicking  the  classroom 
experience  will  preserve  the  status  quo,  while  limiting  the  exploration  of  more  innovative  learning  strategies  that  can 
better  engage  online  learners.  This  status  quo  was  evident  in  that  many  instructors  lacked  innovative  uses  of 
pedagogical  and  technological  techniques  in  their  online  courses. 

There  was  a great  variance  among  online  instructors  on  the  degree  of  facilitating  online  discussion.  This 
study  revealed  that  the  variance  seemed  to  associate  with  a variety  of  factors  (e.g.  timing,  course  type,  perceptions, 
moderating  skills,  etc.).  There  is  a need  to  further  explore  how  and  to  what  degree  each  of  these  contextual  factors 
affects  instructor  presence  in  online  discussion.  From  this  study,  there  was  also  an  issue  as  to  when  to  intervene  in 
online  discussion.  For  example,  when  some  instructors  found  that  their  presence  affected  discussion  flow,  they 
completely  withdrew  from  the  discussion.  While  instructors  in  this  study  rightly  realized  that  their  transition  from 
“dictator”  or  “lecturer”  to  “facilitator,”  they  need  more  concrete  ideas  and  strategies  on  how  to  facilitate  in  a way 
that  does  not  interfere  too  much  with  student  learning  but  yet  contributes  to  content  expertise  and  skill. 

The  findings  related  to  instructors’  perceptions  of  the  social  roles  were  highly  similar  to  those  of  Conrad’s 
study.  As  her  work,  there  was  a general  lack  of  active  social  role  in  facilitating  a learning  community  and  limited 
awareness  of  the  possible  contributions  of  the  social  roles  to  online  learning.  Correspondingly,  our  surveys  from 
students  also  revealed  that  the  level  of  social  presence  in  online  courses  was  low.  Though  the  importance  of  social 
dimension  of  online  learning  has  been  increasingly  acknowledged  in  the  literature,  there  is  still  a gap  between  the 
promises  and  the  reality  of  existing  practices  in  many  online  programs.  This  gap  lies  in  the  change  of  the 
fundamental  conceptions  of  social  roles.  To  assist  with  this  change,  there  is  a need  for  online  instructors  to  be  very 
explicitly  educated  in  the  importance  of  social  roles  and  provided  with  convincing  evidence  of  the  relationship 
between  social  roles  and  cognitive  learning.  Data  from  this  study  indicated  a need  to  further  explore  the  relationship 
between  an  active  social  role  on  the  part  of  the  online  instructor  and  learner  engagement. 

Both  roles  of  “conference  manager”  or  “organizer”  emphasize  the  importance  of  “structure”  (e.g., 
procedural  rules,  clear  timeline,  and  clear  directions)  for  maintaining  disciplined  participation  of  online  learners. 
Consistent  with  previous  studies  (Heuer  & King,  2004;  Pincas,  1998),  this  study  supported  the  notion  that 
instructors  perceived  the  “structure”  to  be  an  essential  condition  for  providing  a psychologically  safe  learning 
environment  and  sustaining  continuous  engagement  in  online  learning. 

The  findings  indicate  that  instructors’  technical  roles  evolved  from  assisting  with  technical  issues  to 
identifying  or  developing  online  pedagogical  tools  to  address  ethical  concerns  of  accessibility.  Consistent  with  prior 
studies  (Andreson  et  al.,  2001;Concrad,  2004),  the  general  technology  used  in  this  online  MBA  program  did  not 
present  significant  challenges  to  online  instructors.  Over  time,  the  instructors  reported  that  the  technology  became 
increasingly  reliable  and  that  the  technical  support  from  the  organization  had  become  more  efficient.  The  emerging 
role  of  “media  designer/developer”  suggests  the  increased  proactive  attitude  of  online  instructors  to  participate  in 
designing  and  developing  new  pedagogical  tools.  While  there  was  a sense  of  excitement  and  adventure  as  instructors 
seemed  to  be  willingly  utilizing  and  learning  about  new  and  emerging  online  technologies,  the  varying  technical 
skills  and  lack  of  knowledge  of  best  practices  in  technology  integration  remained  a key  issue. 

We  also  noted  that  the  tension  of  accessibility  and  sophisticated  use  of  highly  interactive  learning  tools  is  a 
significant  and  growing  challenge  for  technology  integration  in  online  learning.  We  found  that  when  instructors 
were  trying  to  find  the  common  denominator  between  high  interactivity  and  accessibility,  they  often  opted  to  play  it 
safe  and  try  simple  tools  and  activities.  Many  practical  reasons  can  be  contributed  to  their  conservative  choices  (e.g. 
lack  of  time,  lack  of  extensive  training,  the  rapid  growth  of  this  program,  etc.).  However,  the  lack  of  innovativeness 
or  risk  taking  of  the  instructor  seemed  to  be  one  of  the  key  underlying  factors.  There  is  a need  for  organizational 
efforts  to  provide  training  and  policies  which  reward  online  innovation  in  order  to  achieve  a higher  level  of 
technology  use. 
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Time  management  was  identified  as  one  of  the  factors  that  significantly  affected  the  roles  engaged  in  and 
caused  significant  tensions  between  different  roles  (e.g.,  employing  a social  role  or  a pedagogical  role).  This  study 
reinforced  the  previous  finding  that  time  was  consistently  ranked  as  one  of  the  top  barriers  for  teaching  online 
(Berge  & Muilenburg,  2001).  This  issue  indicated  not  only  a need  for  online  instructors  to  adopt  efficient  strategies 
to  teach  online  but  also  a need  to  learn  how  to  strategically  unbundle  some  roles  to  ease  the  stress  (Howell,  Saba, 
Lindsay,  & Williams,  2001).  For  example,  delegating  students  to  play  moderator  roles  in  online  discussions  or  using 
virtual  teaming  strategies  not  only  can  reduce  the  workload  of  online  instructors  but  also  can  enhance  learner 
accountability  while  deepening  their  engagement.  Inviting  outside  experts  can  enhance  the  interactivity  of  an  online 
course  with  opportunities  for  real  world  experience. 


Conclusion 

Exploration  of  instructors’  roles  across  instructors  in  this  study  suggested  that  the  interplay  among  online 
instructor  roles  is  very  complex.  Instructors  in  this  growing  program  have  been  successfully  playing  important  roles 
and  providing  a satisfactory  experience  for  online  learners.  However,  it  was  noted  that  there  was  extensive  variance 
among  online  instructors  on  how  they  play  or  perceive  their  roles  in  this  program.  This  study  also  indicated  that 
several  prominent  contextual  factors  affected  their  roles.  And  there  are  perceived  tensions  between  different  roles.  It 
was  encouraging  to  see  online  instructors  strongly  emphasize  pedagogical  roles  to  assure  quality  online  education. 
However,  there  was  also  an  urgent  need  to  raise  the  awareness  of  the  importance  of  social  roles  if  they  are  to 
establish  a more  engaging  and  fruitful  online  learning  environment. 

To  help  online  instructors  make  a successful  transition  for  online  teaching,  data  from  this  study  indicated  a 
need  for  institutions  to  not  only  plan  future  roles  but  also  provide  substantial  training  support  and  best  practices 
regarding  implementing  those  roles  (Howell  et  al.,  2001).  As  a program  level  case  study,  generalizations  from  the 
results  in  this  study  may  be  constrained.  However,  the  analysis  of  emerging  roles,  issues  and  recommendations 
raised  in  this  study  provide  highly  valuable  information  and  insights  to  assist  distance  educators  and  policy  makers 
to  make  educational  policies  and  practices  for  a successful  transition  for  online  instructors. 
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Background  of  the  Study 

Online  teaching  has  become  a more  established  practice  in  higher  education.  However,  the  quality  of  online 
teaching  is  still  problematic  and  traditional  faculty  development  activities  have  limited  impact  on  faculty  teaching. 

In  this  study,  an  Online  Teaching  Case  Library  (OTCL)  was  proposed  as  a resource  to  improve  faculty  online 
teaching.  This  case  library  is  a repository  of  faculty  online  teaching  courses  and  stories,  which  provide  vicarious 
online  teaching  experiences  to  professors.  The  development  and  validation  of  a solution  like  this  tends  to  be  a long- 
term research  project  requiring  a series  of  studies  and  multiple  research  strategies.  Before  making  any  substantial 
commitment  to  developing  such  a tool,  I conducted  an  exploratory  study  to  identify  the  initial  evidence  to  support 
this  solution. 

Purposes  and  Research  Questions 

The  purposes  of  the  study  are  twofold.  The  first  purpose  is  to  determine  initial  support  for  or  evidence 
against  this  solution  by  exploring  faculty  perceptions  of  a case  library  prototype.  This  focus  may  help  researchers 
and  stakeholders  of  faculty  development  determine  whether  to  pursue  this  solution  in  improving  faculty  online 
teaching.  Assuming  an  OTCL  is  worth  pursuing,  the  second  purpose  of  this  study  is  to  generate  design  knowledge, 
including  a set  of  high-level  design  guidelines  for  future  development  work  in  the  similar  context  and  a 
methodology  on  how  to  develop  a case  library.  As  I mentioned  in  the  previous  paragraph,  this  study  is  the  beginning 
piece  of  study  in  a long-term  research  agenda.  Design  knowledge  synthesized  from  this  study  may  enlighten  future 
research  in  this  or  similar  projects. 

Four  research  questions  were  asked  to  guide  the  study:  (a)  How  do  faculty  members  perceive  a case  library 
as  a tool  that  supports  online  teaching?  (b)  What  tasks  do  faculty  members  perceive  that  they  would  accomplish  in  a 
case  library  that  supports  online  teaching?  (c)  What  types  of  content  do  faculty  members  perceive  that  they  would 
need  in  a case  library  that  supports  online  teaching?  (d)  What  major  system  features  do  faculty  members  perceive 
that  they  would  need  in  a case  library  that  supports  online  teaching?  For  each  research  question,  there  were  two 
subquestions:  (a)  Is  there  a difference  among  faculty  with  different  amounts  of  online  teaching  experience?  (b)  Is 
there  a difference  among  faculty  with  different  levels  of  familiarity  with  case  methods? 

Theoretical  Framework 

The  use  of  case  libraries  to  enhance  faculty  online  teaching  is  supported  by  theories  and  research  in  case 
methods  (Merseth,  1996)  and  case-based  reasoning  (CBR)  (Kolodner,  1993).  Research  shows  that  teacher 
knowledge  is  context- specific  and  situation-dependent  (Calderhead,  Clark  & Peterson,  Clark  & Yinger  as  cited  in 
Merseth,  1996).  Teachers  operate  more  from  “induction  from  experiences”  rather  than  “deduction  from  theoretical 
principles”  (Merseth,  1996,  p.  724).  Therefore,  it  is  important  to  provide  direct  and  vicarious  experiences  to  faculty 
to  enhance  their  teaching.  Cases  have  been  used  in  various  areas  of  teacher  education  (Merseth,  1996),  and  case 
libraries  have  been  developed  to  provide  case  studies  on  technology  integration  (Krueger,  Boboc,  & Cornish,  2003; 
Wang,  Moore,  Wedman,  & Shyu,  2003). 

CBR  (Kolodner,  1993)  is  another  theory  that  supports  the  use  of  case  studies  in  teacher  education.  CBR 
describes  how  human  beings  rely  on  concrete  past  experiences  to  solve  problems.  It  emphasizes  that  vicarious 
experiences  should  be  provided  to  support  problem  solving.  Researchers  in  this  field  have  built  computer-based  case 
libraries  to  support  teaching  and  learning  (Chandler,  1994;  Domeshek  & Kolodner,  1992). 

Methodology 

To  guide  this  study,  I followed  a three-component  methodology  (Ma  & Harmon,  2005):  development 
research,  rapid  prototyping,  and  qualitative  methods.  These  components  describe  the  methods  for  this  study  at  three 
different  levels.  In  the  following  paragraphs,  I present  the  rationale  for  selecting  these  methods  and  discuss  how  they 
guided  this  study. 

Development  Research 

Development  or  design  research  (Reeves,  Herrington,  & Oliver,  2004;  The  Design-Based  Research 
Collective,  2003)  is  an  emerging  research  paradigm  usually  adopted  for  the  dual  purposes  of  addressing  practical 
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problems  and  developing  context-rich  theoretical  knowledge.  This  study  has  both  a practical  goal  of  addressing  a 
faculty  development  issue  and  a theoretical  goal  of  constructing  design  principles  for  an  OTCL.  Therefore, 
development  research  is  an  appropriate  methodology  for  this  study. 

Development  research  provides  guidance  on  how  a research  project  can  be  carried  out  from  the  long-term 
perspective.  Literature  on  development  research  (Cobb,  Confrey,  diSessa,  Lehrer,  & Schauble,  2003;  Collins, 

Joseph,  & Bielaczyc,  2004;  Reeves  et  al.,  2004;  Richey,  Klein,  & Nelson,  2003)  prescribes  that  these  research 
projects  should  involve  (a)  both  development  and  research  activities,  (b)  an  iterative  process  and  a long-term 
engagement;  (c)  collaboration  between  researchers  and  practitioners. 

Development  research  provides  high  level  guidance  on  how  a project  can  be  carried  out.  However,  it  does 
not  prescribe  detailed  procedures  on  how  to  proceed  with  the  project.  This  need  can  be  met  by  the  use  of 
instructional  development  models  (Gustafson  & Branch,  1997).  Rapid  prototyping  is  one  type  of  these  models. 

Rapid  Prototyping 

Rapid  prototyping  is  an  appropriate  development  model  for  this  study.  First,  rapid  prototyping  is  suitable 
for  situations  where  complex  factors  make  it  difficult  to  predict  the  project  outcome.  Technology-based  innovative 
learning  environments  are  usually  complex  and  many  factors  are  involved  in  developing  them  (Hannafin,  Oliver, 
Hill,  & Glazer,  2003;  Reeves  et  al.,  2004;  Reigeluth,  2003).  In  this  project,  there  are  many  multifaceted  issues  in 
developing  a case  library.  Some  of  the  issues  include  stakeholder  needs  and  requirements,  user-interface  design, 
system  technical  design,  as  well  as  the  diffusion  and  adoption  of  these  systems.  Compared  to  traditional  instructional 
design  models,  rapid  prototyping  can  better  handle  such  complexity.  Second,  rapid  prototyping  is  especially 
applicable  in  situations  where  there  is  limited  experience  to  inform  the  design  process.  The  development  of  an 
OTCL  is  such  a case.  Third,  rapid  prototyping  is  appropriate  when  the  development  tools  offer  modularity  and 
plasticity.  An  OTCL  is  computer-based,  so  rapid  prototyping  should  be  appropriate  for  its  development. 

Rapid  prototyping  provides  a process  view  of  how  individual  iterations  progressively  develop  a problem 
solution  and  gradually  build  a knowledge  base.  I synthesized  a rapid  prototyping  model  (Figure  1 and  Figure  2) 
based  on  the  works  of  Tripp  and  Bichelmeyer  (1990)  as  well  as  Dorsey,  Goodrum,  & Schwen  (1997).  This 
model  shows  that  the  study  reported  in  this  paper  is  the  first  iteration  of  a long-term  research  project.  It  also  helps 
structure  the  development  and  research  procedure  for  the  current  study.  There  are  three  major  steps  in  the  current 
study:  conceptualization,  development,  and  research. 


Long-Term  Research 


Iteration  2 


Iteration  3 


Iteration  n 


Figure  1.  Current  study  from  the  long  term  perspective,  adapted  from  Dorsey,  Goodrum,  & Schwen  (1997)  as 
well  as  Tripp  and  Bichelmeyer  (1990). 

At  the  conceptualization  stage,  I first  identified  the  research  problem  and  research  questions.  Then,  I 
synthesized  a problem  solution  from  the  literature  and  developed  this  solution  into  conceptual  models  that  describe 
(a)  the  types  of  tasks  faculty  would  accomplish  in  an  OTCL,  (b)  the  types  of  content  faculty  would  need,  and  (c)  the 
types  of  features  should  be  available.  These  three  models  corresponded  to  three  of  the  research  questions. 

During  the  development  phase,  I implemented  the  conceptual  models  in  a prototype  and  addressed  a variety 
of  issues  involved  in  prototype  development.  This  prototype  primarily  includes  two  components.  One  is  a case  study 
on  the  design  and  delivery  of  an  online  instructional  design  course.  It  contains  an  overview  of  the  course  design  and 
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delivery,  course  materials,  as  well  as  the  lessons  that  the  instructor  learned  from  teaching  this  course.  The  other 
component  deals  with  a specific  topic  in  online  teaching.  These  two  components  represent  two  types  of  content  in  an 
OTCL.  They  allow  the  user  to  explore  an  OTCL  from  the  perspectives  of  examining  similar  courses  or  investigating 
specific  issues. 


Pilot  Study  Formal  Study 


Figure  2.  Development  and  research  procedures  for  the  current  study. 

Some  of  the  development  issues  I addressed  include  scope  of  the  prototype,  tools  and  procedure  used  to 
develop  the  prototype,  content  selection  and  index  selection.  I limited  the  scope  of  the  prototype  to  two  common 
scenarios.  I used  HTML  to  create  a Web-based  medium- fidelity  prototype  (Leone,  Gillihan,  & Rauch,  2000;  Snyder, 
2003).  To  develop  the  prototype,  I followed  four  steps  synthesized  from  several  interface  development  procedures 
(Ludolph,  1998;  Mayhew,  1999;  Weinschenk,  Jamar,  & Yeo,  1997):  (a)  chose  the  object-oriented  design  as  the  basic 
interaction  paradigm;  (b)  developed  primary  user  scenarios;  (c)  identified  the  objects  and  user  actions  (Weinschenk 
et  al.,  1997)  in  the  scenarios;  (d)  developed  the  individual  screens  and  major  navigational  pathways.  The  content  for 
the  prototype  came  from  books,  journal  articles,  and  content  experts.  Indexing  dimensions  and  values  for  cases  are 
based  on  the  indexing  language  of  an  existing  case  library  (Wang  et  al.,  2003),  Gagne’s  taxonomy  of  learning 
outcomes  (1985),  Jonassen’s  typology  of  problem  solving  (2000),  and  Molenda’s  (as  cited  in  Reigeluth,  1999)  list  of 
instructional  methods. 

At  the  research  phase,  I conducted  a pilot  study  to  refine  and  improve  the  research  procedure.  I then  carried 
out  a formal  qualitative  study  to  answer  the  research  questions,  identify  issues  and  possible  improvements  for  the 
solution.  In  this  study,  qualitative  methods  guided  data  gathering  and  analysis.  The  next  section  elaborates  on  these 
methods. 

Qualitative  Methods 

Qualitative  methods  informed  the  data  collection  and  analysis  process  of  the  current  study.  Qualitative 
methods  seem  to  be  especially  helpful  during  the  exploration  phase  of  a development  research  project  (Kelly,  2004; 
Shavelson,  Phillips,  Towne,  & Feuer,  2003)  when  models  (Sloane  & Gorard,  2003),  conjectures  (Sandoval,  2004), 
or  hypothesis  (Kelly,  2004)  are  formulated  in  the  context  of  real  world  problems  and  interventions.  This  study 
explores  faculty  perceptions  of  an  OTCL,  so  qualitative  methods  are  appropriate. 

With  the  purposeful  sampling  technique,  I recruited  seven  faculty  participants  who  had  different 
backgrounds  and  a range  of  experience  related  to  online  teaching  and  case  based  pedagogy.  The  data  collection 
procedure  (Figure  3)  includes  three  stages:  initial  interviews,  contextual  interviews,  and  final  interviews.  The  initial 
interview  examines  faculty  experiences  with  online  teaching  and  case-based  pedagogy,  which  provide  a context  for 
reflective  exploration  of  the  prototype.  The  contextual  interview  (Beyer  & Holtzblatt,  1998;  Holtzblatt  & Jones, 

1993;  Kensing  & Blomberg,  1998)  is  an  ethnographic  field  method  in  systems  design.  In  this  study,  it  involves 
observing  and  interviewing  faculty  participants  while  they  were  interacting  with  the  conceptual  models  and  the 
prototype.  It  consists  of  four  steps.  The  first  step  was  concept  introduction  and  initial  feedbacks.  The  participants 
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were  introduced  to  the  conceptual  models  and  they  discussed  their  reactions  to  the  models.  The  second  step  was 
scenario  review.  The  participants  reviewed  a scenario,  discussed  their  reactions,  and  talked  about  a similar 
experience  they  had.  Discussions  of  their  own  experiences  help  enrich  and  personalize  the  scenario.  In  the  third  step, 
prototype  exploration,  the  participants  explored  the  prototype  using  the  scenarios.  During  the  process,  I asked  the 
participants  questions  in  order  to  understand  their  thought  process,  expectations,  as  well  as  likes  and  dislikes.  The 
last  step  was  prototype  walkthrough.  I walked  participants  through  the  unexplored  features  and  asked  for  feedback. 
Steps  two  to  four  were  repeated  for  the  second  scenario.  The  final  interviews  investigated  faculty  overall 
perceptions  of  an  OTCL  after  their  interactions  with  an  OTCL. 

I consulted  the  literature  on  qualitative  research  to  guide  my  data  analysis.  Two  primary  sources  came  from 
the  works  of  Miles  and  Huberman  (1994)  as  well  as  LeCompte  and  Schensul  (1999).  I transcribe  video  and  audio 
data  and  combined  them  to  generate  transcripts.  Then,  I coded  the  transcripts  into  conceptual  chunks  and  grouped 
them  into  categories.  I drew  flow  charts  to  display  and  make  sense  of  the  relationship  among  the  categories.  Finally, 
I wrote  up  conclusions  and  verified. 


Figure  3.  Data  gathering  procedure. 


Findings 

This  study  intends  to  answer  four  research  questions  related  to  faculty  overall  perceptions  of  an  OTCL  as 
well  as  their  perceptions  of  the  tasks,  content,  and  features  that  this  case  library  should  support.  The  following 
presents  answers  to  these  four  questions  and  discusses  the  findings  in  the  context  of  the  literature. 

Overall  Perceptions  of  an  OTCL 

Faculty  members’  perceptions  of  an  OTCL  focus  on  their  decision  to  use  this  tool.  Figure  4 shows  that 
professors’  perceived  decision  to  use  an  OTCL  can  be  explained  by  three  main  factors:  (a)  perceptions  of  how  an 
OTCL  supports  the  way  they  learn  to  teach  (b)  perceived  usefulness,  and  (c)  perceived  usability  of  an  OTCL.  For 
the  ease  of  communication,  this  figure  is  called  Model  of  Perceived  Decision  to  Use  an  OTCL  (MPDUO). 

Faculty  participants  in  this  study  learned  to  teach  from  trial  and  error  and  from  the  experiences  of  other 
faculty  members.  They  believe  that  an  OTCL  could  support  this  type  of  learning.  It  could  serve  as  an  alternative  to 
human  mentors.  Moreover,  this  tool  could  offer  them  a variety  of  perspectives  and  provide  them  with  experiential 
knowledge  at  the  time  when  they  need  it. 

However,  faculty  might  not  adopt  an  OTCL  unless  it  is  perceived  as  useful  and  usable.  Usefulness  consists 
of  two  dimensions:  applicability  and  relevance.  An  OTCL  should  be  applicable  in  the  sense  that  it  supports  the  tasks 
that  professors  want  to  achieve  during  both  course  design  and  delivery,  and  it  meets  the  needs  of  faculty  who  have 
different  amounts  of  experience  and  preferences.  This  is  the  factor  that  has  driven  participants’  perceptions  of  the 
types  of  tasks  that  they  would  accomplish  with  the  use  of  an  OTCL.  Relevance  is  another  dimension  of  usefulness.  It 
refers  to  instructors’  requirement  that  all  the  resources  related  to  their  tasks  should  be  available  in  an  OTCL, 
regardless  of  whether  they  are  related  to  pedagogy,  content,  or  technical  solutions.  This  has  influenced  faculty 
perceptions  of  the  types  of  content  they  would  needed  in  an  OTCL. 

Usability  includes  two  dimensions:  effectiveness  and  efficiency.  An  OTCL  should  be  effective  in  the  sense 
that  it  should  provide  a shared  language  for  the  user  to  communicate  with  the  tool.  Efficiency  is  another  important 
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dimension  of  usability.  Faculty  wanted  to  quickly  access  the  content  to  carry  out  their  tasks.  This  need  is  reflected  in 
their  requirements  for  appropriate  information  presentation  and  organization  features.  Both  effectiveness  and 
efficiency  are  non-functional  features  faculty  would  need. 
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Figure  4.  Model  of  perceived  decision  to  use  an  OTCL  (MPDUO). 

Faculty  members  with  different  amounts  of  online  teaching  experience  varied  in  their  overall  perceptions  of 
an  OTCL.  Compared  to  novice  online  instructors,  experienced  online  instructors  better  perceived  the  match  between 
an  OTCL  and  professors’  apprenticeship  approach  to  learning  to  teach.  They  also  had  more  detailed  vision  of  how 
an  OTCL  could  support  online  teaching.  Novice  online  instructors,  however,  were  more  concerned  of  the  usefulness 
and  usability  of  an  OTCL,  and  they  were  more  forthright  in  pointing  out  that  they  would  not  use  an  OTCL  unless  it 
could  meet  their  needs.  Their  concern  might  be  explained  by  the  fact  that  the  prospect  of  teaching  online  is  already  a 
challenge  for  novice  online  instructors  and  they  would  be  pressed  for  time  to  put  together  a course;  the  idea  of 
having  to  learn  to  use  another  tool  in  order  to  teach  online  can  add  to  the  stress.  Despite  their  concerns,  however, 
novice  online  instructors  became  more  positive  toward  an  OTCL  once  they  had  more  experience  with  an  OTCL. 
Faculty  members  with  different  levels  of  familiarity  with  case  methods  did  not  seem  to  have  different  overall 
perceptions  of  an  OTCL. 

Perceptions  of  the  Tasks  Faculty  Would  Accomplish  in  an  OTCL 

The  tasks  that  participants  perceived  that  they  would  carry  out  in  an  OTCL  can  be  categorized  as  three 
primary  tasks  and  two  secondary  tasks  (Figure  5).  The  primary  tasks  include  exploring  possibilities,  discovering 
issues,  and  identifying  problem  solutions.  They  are  the  reasons  for  professors  to  use  an  OTCL  during  course  design 
and  delivery.  The  secondary  tasks  consist  of  contributing  to  an  OTCL  and  identifying  the  associated  technical 
solutions.  The  need  for  completing  these  tasks  would  naturally  arise  as  the  user  performs  the  secondary  tasks. 

Participants  with  different  amounts  of  online  teaching  experience  had  different  perceptions  of  the  tasks. 
Novice  online  instructors  tended  to  focus  on  exploring  possibilities  and  identifying  issues  in  online  teaching, 
whereas  more  experienced  instructors  were  apt  to  use  an  OTCL  to  identify  solutions  to  specific  problems.  In 
addition,  I expected  that  experienced  online  instructors  might  be  more  interested  in  adding  stories  and  comments 
than  novice  online  instructors.  There  was  some  evidence  to  support  this  assumption.  Participants  with  different 
levels  of  familiarity  with  case  methods  did  not  seem  to  have  different  perceptions  of  the  tasks. 
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Figure  5.  Evolved  task  model. 


Perceptions  of  the  Types  of  Content  Faculty  Would  Need  in  an  OTCL 

The  types  of  content  that  participants  perceived  that  they  would  need  in  an  OTCL  include  primary  types  of 
content  and  secondary  types  of  content  (Figure  6).  The  primary  types  of  content  are  composed  of  cases  and  topics.  A 
case  has  a case  background,  case  details,  and  lessons  learned.  Case  details  consist  of  learning  outcomes,  teaching 
strategies,  and  course  effectiveness.  A topic  is  represented  by  guidelines  and  the  stories  that  exemplify  the 
guidelines.  Stories  come  from  the  lessons  learned.  The  secondary  types  of  content  refer  to  user  stories  and 
comments,  as  well  as  technical  resources. 


Figure  6.  Evolved  content  model. 

A little  evidence  shows  that  participants  with  different  amounts  of  online  teaching  experience  might 
perceive  the  content  types  a little  differently.  One  participant  perceived  that  novice  online  instructors  would  be  more 
interested  in  cases,  and  topics  would  be  more  pertinent  to  experienced  online  professors.  Participants  with  different 
levels  of  familiarity  with  case  methods  did  not  seem  to  have  different  perceptions  of  the  content. 
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Perceptions  of  the  System  Features  Faculty  Would  Require  in  an  OTCL 

The  system  features  that  faculty  members  perceived  that  they  would  need  fall  into  the  categories  of 
functional  and  non- functional  features  (Figure  7).  Functional  features  are  services  that  an  OTCL  should  provide  to 
enable  professors  to  accomplish  their  tasks,  and  non-functional  features  describe  system  properties  with  regard  to 
how  well  the  system  provides  the  functional  features. 


* The  arrow  indicates  that  a type  of  features  includes  sub-types  of  features. 

Figure  7.  Evolved  conceptual  model  of  features. 

Functional  features  faculty  perceived  that  they  would  require  can  be  classified  as  content  access  features 
and  user  contributions  features.  Content  access  features  consist  of  those  that  give  access  to  the  primary  types  of 
content  and  secondary  types  of  content  as  well  as  those  that  provide  internal  or  external  links.  Case  browse,  topic 
browse,  case  search,  and  topic  search  are  four  content  access  features  that  provide  access  to  the  primary  types  of 
content;  links  to  technical  resources  and  access  to  user  stories  and  comments  are  two  content  access  features  enables 
the  retrieval  of  secondary  types  of  content;  case  to  topics,  topics  to  cases,  and  summaries  to  details  are  three  content 
access  features  that  gives  users  flexibility  to  navigate  among  different  types  of  content  in  and  out  of  an  OTCL.  In 
addition  to  content  access  features,  the  other  type  of  functional  features  is  user  contribution  features,  which  are 
composed  of  add  stories  and  add  comments. 

Non-functional  features  that  participants  considered  as  important  are  comprised  of  two  usability 
dimensions:  effectiveness  and  efficiency.  These  two  features  suggest  that  a variety  of  language  issues  as  well  as 
information  presentation  and  organization  issues  be  addressed. 

Participants  with  different  amounts  of  online  teaching  experience  perceived  some  features  differently. 
Compared  to  experienced  online  instructors,  novice  online  instructors  were  more  concerned  with  effectiveness  and 
efficiency  and  would  not  tend  to  add  stories  or  comments  to  the  tool.  Participants  with  different  levels  of  familiarity 
with  case  methods  did  not  seem  to  have  different  perceptions  of  the  features. 
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Implications  of  Study 

The  two  purposes  of  the  study  were  fulfilled.  This  project  identified  the  initial  evidence  to  support  an 
OTCL  as  an  online  teaching  resource.  Faculty  might  use  an  OTCL,  because  it  matched  the  way  that  they  learned  to 
teach.  However,  there  were  many  challenges  involved  in  making  the  tool  useful  and  usable.  This  study  would 
provide  a base  for  decision  makers  to  determine  whether  they  would  adopt  this  tool. 

For  those  who  do  want  to  pursue  this  solution,  a set  of  high-level  design  guidelines  and  a methodology  on 
how  to  develop  an  OTCL  were  developed  based  on  the  findings  of  the  study.  These  design  guidelines  include  (a) 
enhance  the  perception  that  an  OTCL  supports  the  way  faculty  learn  to  teach,  (b)  enhance  perceived  usefulness  of  an 
OTCL,  and  (c)  enhance  the  perceived  usability  of  an  OTCL.  A methodology  on  how  to  develop  a case  library  has 
evolved  from  this  study.  This  methodology  consists  of  three  components:  development  research,  rapid  prototyping, 
and  qualitative  methods.  Development  research  describes  the  nature  of  this  methodology;  rapid  prototyping  frames 
the  development  and  research  process;  qualitative  methods  with  the  use  of  contextual  interviews  may  guide  data 
gathering  and  analysis. 

In  addition  to  attaining  the  two  purposes,  this  study  has  contributed  to  the  following  overlapping  areas  of 
theories  and  research:  challenges  of  online  teaching  for  professors,  faculty  change  and  teaching  improvement, 
faculty  needs  in  online  teaching,  electronic  performance  support  systems  (EPSS),  knowledge  management  systems 
(KMS),  technology  acceptance,  as  well  as  case-based  reasoning  and  case  methods. 
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Understanding  and  Representing  Learning  Activity  to  Support  Design: 

A Contextual  Design  Example 

Mark  Notess 
Indiana  University 

Abstract 

Contextual  Design  is  a well-defined  method  for  gathering  and  representing  user  understanding  within  a 
coherent  design  process.  This  paper  illustrates  the  value  of  Contextual  Design  to  educational  system  design  by 
describing  a case  study  involving  14  contextual  inquiry  sessions  that  were  carried  out  in  and  around  the  Indiana 
University  music  library,  in  support  of  designing  technology  for  student  learning.  Findings  are  presented  as 
diagrammatic  work  models,  an  affinity  diagram,  and  a list  of  insights  and  design  ideas.  The  utility  of  Contextual 
Design  work  modeling  is  assessed,  and  some  limitations  in  its  ability  to  represent  learning  are  considered. 

Introduction 

Designing  useful  educational  technologies  requires  us  to  understand  the  potential  users  of  those 
technologies,  the  tasks  the  users  bring  with  them,  and  the  contexts  of  use.  As  we  have  developed  our  next-generation 
digital  music  library,  Variations2  (Variations2,  2005),  we  have  taken  a valuable  opportunity  to  gain  this 
understanding  by  studying  usage  of  its  predecessor,  the  Variations  system  (Dunn  & Mayer,  1999;  Variations,  2005). 
This  paper  reports  on  the  second  in  a sequence  of  contextual  inquiry  studies  of  digital  music  library  use.  Results 
from  the  first  study  have  been  partially  reported  (Notess,  2004a).  A small  subset  of  results  from  the  present  study 
were  reported  elsewhere  (Notess,  2004b). 

Contextual  inquiry  (Holtzblatt  & Jones,  1993)  is  a naturalistic  inquiry  research  method  wherein  researchers 
observe  actual  work  in  normal  work  contexts.  The  method  is  intrusive  in  that  the  observer  sits  with  the  participant 
and  asks  clarifying  questions  during  and/or  following  the  session.  An  advantage  of  contextual  inquiry  over  an 
interview  or  questionnaire  protocol  is  that  it  gets  beyond  the  participants’  beliefs  about  their  activity  and  records 
actual  practice,  which  often  differs  from  how  participants  believe  they  behave.  Data  from  contextual  inquiry  has 
proven  very  useful  in  information  systems  design.  Recently,  it  has  been  applied  to  the  design  of  educational 
technologies  as  well. 

Contextual  Design  (Beyer  & Holtzblatt,  1998)  extends  contextual  inquiry  by  situating  contextual  data 
within  a coherent  design  process.  Central  to  the  design  process  are  five  types  of  diagrammatic  “work  models”  which 
serve  to  represent,  consolidate,  and  communicate  the  findings  of  contextual  inquiry  in  a manner  useful  to  technology 
design.  This  paper  illustrates  the  application  of  Contextual  Design  (CD)  to  educational  technology  using  a case 
study  of  digital  music  library  use  at  Indiana  University. 

Motivating  the  study  were  the  following  research  questions. 

• What  is  the  role  of  the  Variations  digital  music  library  in  the  studies  of  music  students?  Into  what 
larger  patterns  of  student  work  does  a digital  music  library  fit? 

• How  do  students  decide  between  using  online  and  physical  resources?  For  online  resources,  what  use 
do  students  make  of  the  non-library  web? 

Beyond  these  primary  questions,  we  were  also  interested  in  characterizing  the  suitability  of  contextual 
inquiry  and  the  CD  work  models  for  examining  and  representing  student  library  work.  The  CD  models  were 
developed  primarily  for  describing  business-oriented  work  activity.  Whether  the  models  adapt  well  to  representing 
learning  activity  is  an  open  question. 

After  describing  the  digital  music  library  technologies  and  the  method  for  conducting  the  observations,  this 
paper  provides  examples  of  the  five  CD  work  models,  describes  the  useful  outcomes  of  the  study  for  Variations2 
design,  and  offers  a preliminary  assessment  of  the  CD  models’  utility  in  studying  learning. 

Study  Description 

Variations  and  Variations2 

Variations  is  a large  digital  music  library  implementation  at  Indiana  University  (IU),  in  use  since  1996  to 
provide  streaming  audio  and  digitized  score  images  to  music  students  at  computers  in  the  music  library.  The 
approximately  1500  School  of  Music  students  make  frequent  use  of  Variations  for  course  reserves  and  personal 
study  although  anyone  with  an  IU  network  login  may  go  to  the  Cook  Music  Library  and  use  Variations. 

The  collection  includes  over  10,000  digitized  recordings.  A Variations  recording  comprises  a CD,  CD  set, 
LP,  LP  set,  tape,  etc.;  hence,  an  item  in  the  library  catalog  maps  to  a single  Variations  item.  The  Variations 
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collection  has  grown  through  successive  semesters  of  course  listening  reserves  being  requested  by  faculty.  For 
copyright  reasons,  access  to  Variations  is  limited  to  PCs  in  the  music  library  on  the  IU  campus  as  well  as  to  network 
ports  in  several  music  classrooms.  Most  usage  of  Variations  occurs  at  the  approximately  90  PCs  in  the  music  library. 
Recordings  in  Variations  are  accessed  via  a URL-based  mechanism.  URLs  for  Variations  recordings  are  embedded 
in  the  online  library  catalog  (IUCAT)  records,  but  URLs  are  also  available  on  HTML-based  course  reserve  pages 
and  can  be  put  in  any  web  page  or  typed  directly  into  a web  browser.  Clicking  on  a Variations  link  or  entering  the 
URL  in  a browser  brings  up  a web  page  summarizing  the  item  and  its  contents.  The  Variations  audio  player  is 
invoked  by  clicking  on  the  a link  to  the  desired  side  (or  CD)  listed  on  the  web  page.  Since  Variations  recording  files 
are  stored  on  tape  and  copied  to  disk-based  cache  on  demand,  a user  may  have  to  wait  several  minutes  for  the  player 
to  appear  if  no  one  has  requested  that  item  during  the  previous  day  or  two. 

Variations2  is  a completely  new  digital  music  library  project  that  is  replacing  Variations  at  IU  (the  actual 
replacement  occurred  in  May,  2005,  subsequent  to  the  present  research).  In  addition  to  library  functionality 
(resource  discovery  and  access),  Variations2  also  provides  pedagogical  tools  to  support,  for  example,  bookmarking, 
playlists,  analytical  and  annotation  tools,  and  listening  drills.  The  research  reported  in  this  paper  was  carried  out  in 
support  of  designing  Variations2  product  features  by  better  understanding  how  students  work  within  the  existing 
Variations  tool  ecology. 

Participants 

Our  first  study  of  Variations  users  had  focused  on  undergraduate  instrumental  performance  majors.  For  this 
study,  we  recruited  four  participants  who  were  graduate  voice  students  in  a song  literature  class.  We  knew  voice 
students  had  some  needs  we  had  not  previously  explored,  such  as  viewing  song  texts,  opera  librettos,  and  their 
translations.  Participants  were  selected  from  among  volunteers  signing  up  during  a class  presentation  about 
Variations2.  The  selected  participants,  three  males  and  one  female,  were  the  first  four  to  reply  to  a follow-up  email. 
Participants  received  a gift  certificate  in  exchange  for  letting  us  observe  and  discuss  approximately  four  to  five  hours 
of  their  academic  activities. 

Method 

Observations  occurred  in  sessions  of  between  one  and  two  hours  in  length.  While  we  were  primarily 
interested  in  their  library  work,  we  also  wanted  to  get  a broader  picture  of  voice  students’  information  needs  in  their 
academic  activities.  Therefore,  we  included  observations  of  other  activities  such  as  voice  lessons  (both  teaching  and 
taking),  class  attendance,  and  ensemble  rehearsal.  In  some  cases,  depending  on  the  nature  of  the  work  being 
observed,  it  was  possible  to  discuss  aspects  of  the  activity  during  the  observation.  For  instance,  during  library  stack 
browsing,  it  was  easy  to  talk  about  what  the  student  was  looking  for  and  why.  During  less  interruptible  activities 
such  as  taking  a class  or  giving  a voice  lesson,  the  observer  saved  up  questions  and  clarifications  for  a subsequent 
discussion  with  the  participant. 

Participants  were  requested  to  do  their  normal  academic  activities  (whatever  they  needed  to  do  next)  and 
were  observed  in  their  usual  contexts — library  computer  carrel  or  work  table,  classroom,  or  lesson  studio.  Data  were 
collected  with  handwritten  notes.  No  audio  or  video  recordings  were  made.  Artifacts,  such  as  assignment  sheets  or 
student  notes,  were  photocopied  and  annotated.  At  end  of  each  observation,  we  talked  with  the  participant  to  make 
sure  we  were  understanding  the  observed  activity  correctly,  to  make  any  needed  photocopies,  and  to  set  up  any 
subsequent  observation. 

Contextual  inquiry  focuses  on  understanding  the  work  being  done,  in  all  its  richness.  To  collect  these  data, 
we  ask  questions  to  be  sure  we  understand  the  triggers  that  initiate  a work  sequence,  the  intent  to  be  fulfilled  by  the 
work  sequence,  the  steps  involved,  the  artifacts  created  or  used  during  the  work,  the  people  communicated  with,  the 
pressures  and  influences  that  impact  the  work,  the  breakdowns  that  interfere  with  the  work,  and  the  environment 
within  which  the  work  occurs  (Beyer  & Holtzblatt,  1998). 

Data  from  these  sessions  were  represented  in  each  of  the  five  Contextual  Design  work  models:  Sequence, 
Flow,  Culture,  Physical  and  Artifact.  Each  type  of  model  was  then  consolidated  across  the  inquiry  sessions  to  yield  a 
consolidated  model.  Consolidated  models  indicate  larger  patterns  without  losing  the  detail  of  individual  variation. 
Contextual  design  also  includes  an  affinity  diagram  process,  where  researchers  and  volunteers  create  an  extensive 
affinity  diagram  of  “work  notes”,  which  can  include  nuggets  of  data  from  the  models  as  well  as  data  that  did  not  fit 
neatly  into  any  of  the  models. 


Findings 

We  conducted  fourteen  contextual  inquiry  sessions.  For  all  of  these  sessions,  our  focus  was  on  the 
information  needs  of  the  participants.  Ten  of  these  were  observations  of  library  work;  the  other  sessions  included  a 
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voice  lesson  for  the  participant,  a voice  lesson  given  by  the  participant  to  a non-major  (called  a “secondary  voice 
lesson”),  a class  session,  and  an  ensemble  rehearsal.  All  sessions  produced  at  least  some  useful  data  with  the 
exception  of  the  ensemble  rehearsal.  This  rehearsal  was  a dress  rehearsal  for  a ballet  that  included  choral 
accompaniment,  but  the  focus  of  the  rehearsal  had  nearly  nothing  to  do  with  the  singers,  who  were  merely 
commended  for  their  excellent  work. 

The  ten  library  sessions  included  observations  of  the  following  kinds  of  work: 

• Listening  assignment.  Students  listen  to  an  assigned  set  of  songs,  or  select  from  among  assigned  songs, 
and  write  brief  analyses  of  what  they  hear  and  think. 

• Recital  assignment.  Students  plan  an  imaginary  voice  recital  following  one  or  more  specific  themes, 
such  as  British  art  songs  related  by  some  thread  such  as  having  the  same  poet  or  same  subject  matter. 

• Audition  “package  ” preparation.  Students  auditioning  for  a summer  singing  job  or  other  opportunity 
select  vocal  material  to  polish  that  will  meet  usual  audition  criteria  for  quantity  and  variety  while  also 
showing  off  the  strengths  of  their  individual  voices. 

• Lesson  piece,  recital  or  performance  part  preparation.  Students  study  a particular  piece  or  part  for 
performance  in  their  lesson,  recital  or  a production.  Study  includes  not  only  listening  to  and/or 
watching  various  performances  but  also  uncovering  background  information  about  the  composer,  the 
poet,  the  performers,  and  so  on.  Study  may  also  include  making  a literal  translation  of  the  text. 

• Song  analysis  project.  Students  perform  an  in-depth  analysis  of  a particular  song,  tracing  its  history 
through  various  performers  and  performances. 

• Exam  preparation.  Students  study  a body  of  work  so  that  they  can  identify  and  discuss  a given  song 
upon  hearing  it  in  an  exam. 

Work  Models 

Models  included  in  this  paper  are  consolidated  work  models  (Beyer  & Holtzblatt,  p.  23):  they  represent 
multiple  observations  of  similar  work.  In  Contextual  Design,  consolidated  models  are  built  by  induction  from 
examining  individual  work  models  (models  representing  a single  observation). 

Sequence  Model.  Nearly  all  the  library  work  consolidated  into  one  of  the  two  sequences  shown  in  Table  1. 
The  two  sequences  share  some  steps  (“intents”  in  CD  terminology).  The  key  difference  between  the  two  types  of 
work  was  twofold:  whether  the  materials  were  pre-identified,  and  the  level  at  which  the  studying  occurred.  For  the 
“Study  in  Detail”  sequence,  the  library  material  to  be  used  was  known  ahead  of  time,  or  was  selected  from  a 
predetermined  list,  and  just  had  to  be  located.  For  the  “Collect  and  Select”  sequence,  the  student  started  with  criteria 
but  not  material.  Finding  material  that  met  the  criteria,  through  a process  of  building  a candidate  pool  and  then 
selecting  from  that  pool,  was  part  of  the  work  itself.  Materials  were  selected  based  on  descriptive  data  (length, 
language,  key,  etc.)  and  a more  cursory  listen  to  the  content  than  in  the  first  sequence.  In  the  first  sequence,  detailed, 
repeated  listening  to  the  content  absorbed  the  attention  and  the  descriptive  data  were  used  more  sparingly  (performer 
name  was  the  most  common). 

Although  our  data  did  not  show  these  two  sequences  in  combination,  it  was  clear  that  some  tasks  begin  as 
“Collect  and  Select”  but  will  later  require  “Study  in  Detail.”  The  song  analysis  project,  mentioned  above,  is  one 
example:  one  first  has  to  select  seminal  performances  of  a piece  and  then  listen  in  detail  to  complete  the 
musicological  analysis. 

The  sequence  model  has  an  additional  level  of  detail  not  shown  here  because  of  space  limitations.  The  lowest  level 
of  detail,  “abstract  steps”,  describes  the  steps  taken  to  accomplish  the  higher-level  intent  in  the  table.  For  instance,  in 
the  “Study  in  Detail”  sequence,  the  “retrieve  known  recording”  intent  has  the  abstract  steps  shown  in  Table  2.  Note: 
In  Table  2 and  elsewhere  in  this  paper,  the  “BD”-prefixed  comments  refer  to  work  breakdowns — observed 
difficulties. 


Table  1.  The  Two  Common  Sequences  of  Library  Work 


Activity 

“Study  in  Detail” 

“Collect  and  Select” 

Prepare  to  do 

• get  headphones 

library  work 

• find  available  carrel 

• locate  assignment 

• log  in  and  locate  on-line  tools 
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• select  piece  to  study 

• retrieve  known  recording 

• retrieve  known  auxiliary  materials  (scores, 
texts,  reference  works) 

Work  with 
library  materials 

• study  material  (listen,  and  follow  along  in 
score  and/or  text;  repeat  whole  piece  or 
key  parts) 

• make  personal  notes  to  capture  key  points 
gleaned  from  studying 

• find  candidate  materials 

• examine  many  details  quickly  to  decide 
which  to  select  (listen,  check  length, 
performer,  key,  etc.) 

• make  personal  notes  to  guide  selection 

• write  assignment  deliverable 

Wrap-up  the 
work 

• preserve  notes  and/or  assignment  deliverable  (email  to  self,  save  on  Zip  disk  or  network 
drive,  print) 

• log  out 

• pack  up 

• return  reserve  materials 

• return  headphones 

Table  2.  Abstract  Steps  for  “Retrieve  Known  Recording” 

• Find  course  reserve  list 

• Scroll  to  desired  recording  (BD:  reserve  list  may  be  very  long) 

• Select  item  (BD:  easy  to  pick  wrong  item  due  to  title  similarities) 

(or) 

• Looking  at  assignment  sheet,  type  Variations  URL  for  item  in  browser  field 
(or) 

• Enter  search  terms  in  online  catalog 

• Scroll  through  search  results  to  find  desired  item  (BD:  easy  to  pick  wrong  item  due  to  title  similarities  and 
lack  of  distinct  visited-link  color) 

• Select  item 

• Select  CD/Side  within  Variations  web  page 


Culture  Model  The  consolidated  culture  model,  which  captures  the  power,  influence,  and  emotional 
dynamics  between  people  and  groups,  is  shown  in  Figure  1.  This  is  not  a particularly  rich  example  of  a culture 
model.  The  relative  paucity  of  culture  data  likely  results  from  the  fact  that  the  work  observed  was  primarily 
individual  study  within  a well-defined  hierarchy  (teacher- student).  If  our  observations  had  focused  on  collaborative 
or  competitive  activities,  more  cultural  forces  would  probably  have  emerged. 

Of  potential  interest  from  this  diagram  is  the  one  breakdown,  shown  in  the  diagram  with  a heavy  zigzag. 
None  of  the  participants  was  willing  to  recall  material  on  loan  to  another  patron.  In  one  case,  the  participant  justified 
his  reluctance  by  saying  that  if  someone  else  had  it  checked  out,  he/she  probably  has  a greater  need  for  it.  This 
situation  illustrates  an  important  limitation  of  physical  materials  and  also  shows  that  at  least  some  parts  of  the 
community  of  music  library  patrons  view  their  use  of  scarce  materials  as  a collaboration  rather  than  competition. 
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Figure  1 . Consolidated  Culture  Model 


Flow  Model  The  flow  model  (Figure  2)  represents  the  movement  of  artifacts  and  communication  between 
people  (or  roles)  to  accomplish  work.  Beneath  each  role  are  listed  the  observed  responsibilities  of  that  role.  The  arc 
labels  in  boxes  represent  tangible  or  virtual  artifacts;  the  other  arc  labels  indicate  communication  not  formalized  in 
an  artifact. 


As  with  the  culture  model,  this  flow  model’s  relatively  simplicity  is  due  to  the  solitary  nature  of  the 
observed  activity.  And  again,  it  is  the  breakdown  that  is  most  interesting.  In  this  case,  we  see  that  material  known 
(or  assumed)  by  one  person  may  be  unfindable  by  another.  Two  causes  of  this  breakdown  were  observed.  First, 
material  may  not  be  readily  searchable  using  the  terms  in  which  people  remember  and  describe  it — looking  for  a 
particular  singer  may  not  work  if  works  are  cataloged  using  the  name  of  the  singer’ s ensemble  rather  than  the 
individual  performer.  Second,  it  can  be  difficult  to  remember  where  materials  reside — are  they  in  the  library  or  not? 
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Physical  Models.  A variety  of  consolidated  physical  models  are  possible,  such  as  the  arrangement  of 
windows  on  a computer  display,  the  library  layout,  or  personal  workspace.  We  show  just  one  of  these.  Figure  3,  the 
workspace,  is  illustrated  with  an  annotated,  mocked-up  photograph  taken  in  the  actual  carrels  used  by  students. 
Usage  of  the  workspace  by  students  is  remarkably  uniform:  students  hold  the  auxiliary  material  (score,  texts)  in 
front  of  them,  type  with  their  arms  lying  across  it,  have  a pile  of  other  papers  on  their  left  (the  right  is  kept  clear  for 
the  mouse  and  the  headphones  cord)  and  use  the  shelves  in  front  of  them  for  other  materials  such  as  reference  books. 
Backpacks  are  usually  on  the  floor  to  the  left  of  the  chair.  The  carrels  are  uniformly  sized,  with  no  collaborative 
spaces  available. 


Figure  3.  Physical  Model:  Library  Computer  Carrel 

We  noted  several  breakdowns  in  the  physical  environment: 

• Headphones  and  headphone  jacks  sometimes  do  not  work  well.  The  lA  inch  jacks  require  students  with 
their  own  headphones  to  buy  and  (more  difficult)  not  lose  a plug  adapter. 

• Sound  quality  is  very  important  to  voice  students.  They  are  also  sensitive  to  background  noise  and  to 
the  sound  leak  from  the  open-ear  headphones  loaned  by  the  library. 

• The  carrels,  although  capacious  by  some  standards,  are  cramped  work  areas. 

• During  busy  periods,  students  have  to  wait  in  line  for  a computer. 

Artifact  Models.  In  fact,  multiple  artifact  models  are  needed:  assignment  sheets,  completed  assignments, 
graded  assignments — these  are  the  main  artifacts.  Space  limitations  preclude  showing  artifact  models.  The  main 
breakdown  associated  with  artifacts  is  that  assignments  were  all  paper-based.  This  introduced  two  difficulties:  if  a 
student  forgets  the  assignment  sheet,  there  is  no  way  to  retrieve  it  on  line;  second,  long  URLs  and  other  text  have  to 
be  retyped,  sometimes  introducing  errors.  The  limited  bibliographic  data  associated  with  Variations  can  be  included 
in  student  deliverables  via  cut-and-paste,  but  the  content  (the  recording)  cannot  easily  be  excerpted  and  used  outside 
its  context. 

Other  Contextual  Design  Outputs 

Affinity  Diagram.  The  CD  process  encourages  capturing  many  “work  notes”  during  modeling  which  are 
later  turned  into  an  affinity  diagram.  The  affinity  diagram  is  hierarchical,  with  the  categories  emerging  from  the  low- 
level  data.  Starting  with  138  total  work  notes,  we  ended  up  with  five  broad  categories  and  one  level  of  subcategories 
(Table  3). 

Work  note  affinity  diagrams  are  approximate,  but  they  provide  a fairly  quick  way  of  organizing  a lot  of 
information  that  does  not  always  fit  neatly  into  the  more  painstakingly  constructed  work  models.  Strangers  to  the 
project  can  use  the  affinity  as  an  easy  way  to  begin  learning  about  at  least  one  category  of  our  users. 

Insights  and  Design  Ideas.  The  purpose  of  collecting  and  analyzing  this  mass  of  data  is  to  arrive  at  useful 
insights  into  users,  tasks,  and  contexts.  Those  insights  in  turn  generate  design  ideas.  Table  4 shows  a sampling  of  the 
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insights  we  gained.  As  the  data  reported  in  this  paper  are  of  necessity  partial,  some  of  the  insights  are  from  data  or 
discussions  not  reported  here.  However,  it  is  useful  to  show  the  full  range  of  insights  obtained. 


Table  3.  Top  Two  Levels  of  Affinity  Diagram 


1 . The  context  I work  in 

a.  I have  to  work  in  a campus  computer  lab 

b.  I have  to  learn  the  library 

c.  Why  I like  Variations 

d.  Problems  I have  with  Variations 

e.  I have  to  deal  with  my  workspace 

f.  How  I manage  my  windows 

g.  I have  to  manage  lots  of  stuff 

h.  I copy  what  I need 

2.  How  I find 

a.  How  I find  my  tools 

b.  I need  the  right  song 

c.  Search  tools  are  clumsy  and  unforgiving 


d.  I have  to  sift  through  results 

e.  I try  to  find  materials  by  browsing 

f.  I use  the  web  to  find 

3.  Physical  vs.  Online  Materials 

a.  Why  I won’t/don’t  use  physical  materials 

b.  Why  I use  physical  materials 

4.  How  I examine 

a.  I need  song  length 

b.  I have  to  assimilate  lots  of  details 

c.  How  I decide  what  to  sing 

d.  How  I prepare  a song 

5.  What  I have  to  do  “for  a piece  of  paper”  [degree] 
a.  [no  subcategories] 
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Table  4.  Insights  and  Design  Ideas 


Insight 

Design  Ideas 

It  takes  too  long  to  find  offline  materials  for  short 
assignments.  (This  is  perception. . .is  it  true?) 

Locate  computer  carrels  close  to  physical  material, 
or  push  wireless  and  put  tables  near  high-use 
materials. 

Finding  things  serendipitously  (by  online  browsing 
or  shelf  browsing — not  equivalent)  is  important. 

Provide  a way  to  browse,  not  just  search,  the 
Variations2  collection. 

The  value  of  peripheral  learning  is  not  always 
realized  by  students. 

Public  computers  in  library  treat  all  users  as  first 
time  users,  offering  them  the  same  configuration 
choices,  etc.,  each  time  they  sit  at  a different 
machine.  This  is  inefficient  but  accepted. 

[This  is  well  beyond  our  scope  but  can  feed  into 
design  of  future  computer  labs  in  the  library.] 

Web  resources  are  very  important,  especially 
Google  and  recmusic.org. 

Provide  links  within  Variations2  bibliographic  data 
to  good  external  resources. 

To  plan  recitals,  users  may  want  to  listen  to  several 
recordings  in  order,  either  from  same  or  different 
CDs. 

Define  comparison  sets  or  collections. 

provide  a way  to  define  a list  of  songs/pieces  from 

same  or  different  containers,  and  listen  to  them  in 

order. 

Students  sometimes  need  to  retrieve  all 
performances  of  a given  piece. 

Allow  search  results  to  list  instantiations 
(performances)  by  a performer,  sorted  by  date. 

Singers  do  many  library  tasks  in  parallel — they 
might  be  listening,  following  along  in  score  or  text, 
taking  notes,  polishing  a deliverable,  and/or 
searching/browsing  for  additional  information  or 
new  material. 

Variations2  should  support  parallel,  not  just  linear, 
activity.  Is  a specialized  window  organizer  needed? 

Students  can  copy  and  reuse  Variations 
bibliographic,  but  the  collection  content  (the 
recordings)  are  not  readily  reusable. 

Bring  up  just  one  song,  not  the  whole  container. 
Provide  a way  to  build  and  share  personal 
collections  of  excerpts.  Variations2  bookmarking 
moves  in  this  direction  but  does  not  allow  excerpt 
specification. 

The  composer’s  key  isn’t  necessarily  the  singer’s 
key. 

For  vocal  works,  indicate  what  keys  work  for 
different  voice  ranges. 

On  the  Variations2  development  team,  we  have  responded  to  the  data  in  different  ways  depending 
on  the  nature  of  the  finding.  For  example,  after  several  observations  demonstrated  how  important  and 
challenging  it  was  for  participants  to  figure  out  song  length,  we  immediately  and  easily  added  track  length 
to  the  end  of  the  track  title  in  the  Variations2  player.  However,  some  ideas  offer  more  fundamental 
challenges.  Providing  a way  to  retrieve  all  performances  of  a piece  by  date  requires  a substantial  rethinking 
of  the  Variations2  search  interface.  It  is  not  a quick  fix,  so  it  will  be  considered  as  a requirement  for  a 
future  release.  Similarly,  including  data  on  what  keys  work  for  different  vocal  ranges  would  require 
additions  to  our  data  model,  changes  to  which  ripple  through  several  parts  of  the  system. 

It  is  the  nature  of  contextual  inquiry  to  generate  data  on  a broad  range  of  concerns,  often  much 
broader  than  we  expect  when  we  begin  the  study.  We  come  up  with  ideas  for  new  tools  or  new  ways  to 
organize  the  library’s  physical  space.  We  pass  the  data  and  ideas  along  to  other  people  who  have  an  interest 
in  the  problems  raised — library  administration  or  faculty. 

Conclusions 

Where  Variations  “Fits” 

Initially,  we  asked  how  Variations  fits  into  the  larger  patterns  of  student  library  work.  An 
examination  of  each  consolidated  model  reveals  the  various  dimensions  of  “fit.” 

Sequence.  Variations  is  the  tool  of  choice  for  “listening  in  detail”  or  doing  the  overview  listening 
needed  for  the  “collect  and  select”  sequence.  Variations  is  also  useful  for  comparing  performances  and 
determining  a piece’s  length. 
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Culture.  Variations  allows  students  to  learn  from  the  “community  of  famous  performers”  more 
extensively  and  efficiently  than  they  would  likely  do  if  they  had  to  use  physical  materials.  But  because  the 
Variations  collection  is  not  comprehensive,  the  “community”  may  be  artificially  limited  based  on  what 
faculty  have  chosen  to  place  on  reserve. 

Flow.  Recordings  are  put  in  Variations  at  faculty  request  and  serve  as  reserve  lists  for  classes.  As  a 
result,  class  assignments  depend  on  Variations,  often  identifying  specific  recordings  by  their  Variations 
URLs.  Yet  whenever  possible,  students  also  rely  on  Variations  for  assignments  that  do  not  identify 
recordings.  Attempts  to  direct  students  to  Variations  material  without  providing  a link  or  URL  are 
potentially  problematic. 

Physical.  Students  can  only  use  Variations  at  public  computers  in  the  music  library.  Thus  they 
must  deal  with  issues  related  to  public  access  computer  stations:  availability,  configuration,  maintenance, 
limited  work  area,  etc.  Because  Variations  is  a Windows-based  application  with  a one-recording,  one 
window  model,  and  because  students  retrieve  information  from  many  online  sources  and  use  word 
processing  tools,  Variations  lives  on  a complex,  crowded  desktop. 

Artifact.  Students  often  access  Variations  starting  from  paper.  Only  the  (limited)  Variations 
bibliographic  data  can  be  included  in  student  deliverables,  not  the  music  itself.  The  content  is  “locked”  in 
Variations;  hence  students  must  describe  the  content  themselves  but  cannot  illustrate  their  description  with 
excerpts.  Content  cannot  be  collected  easily  and  set  aside  for  later  use — it  must  be  retrieved  again. 

Thus  it  has  been  possible  to  determine  where  Variations  fits  by  looking  at  the  broader  context. 
Contextual  inquiry  forces  us  to  lift  our  gaze  beyond  our  small  piece  of  software  and  grapple  with  its  place 
in  a user’s  full  experience.  While  detailed  analyses  of  software  can  uncover  ease-of-use  problems  (cf. 
Blandford  & Stelmaszewska,  2002),  such  analytical  approaches  cannot  tell  us  much  about  usefulness. 

Physical  vs.  Online  Materials 

Deciding  When  to  Use  Which.  For  recordings,  online  was  the  obvious  choice  and  overwhelming 
preference.  Participants  exhibited  visible  dismay  when  the  recording  they  were  interested  in  wasn’t 
available  in  Variations.  The  common  explanation  for  this  preference  was  the  convenience  and  speed  of 
online  access.  Participants  were  unwilling  to  request  physical  recordings  for  short  assignments.  For  larger 
assignments,  participants  indicated  plans  to  request  recordings  from  the  library  or  borrow  recordings  from 
other  sources  (dormitory  CD  libraries,  friends  with  large  CD  collections).  One  participant  spoke  of  plans  to 
burn  needed  songs  onto  a personal  CD  using  the  library’s  CDs  and  music  software.  Apart  from  CDs, 
students  may  be  losing  familiarity  with  recording  formats.  One  participant  found  a 78  RPM  item  in  the 
catalog  but  was  not  familiar  with  that  format. 

For  visual  materials  such  as  scores,  song  texts  or  librettos,  or  reference  works,  preferences  were 
less  clear.  Online  scores  are  still  not  common.  The  only  observed  computer-based  score  usage  was  from  a 
participant’ s personally  purchased  copy  of  a commercial  CD  containing  a score  collection.  Participants 
expected  to  have  to  find  physical  scores  via  the  library  catalog  or  shelf  browsing. 

On  the  other  hand,  song  texts  and  librettos  were  regularly  retrieved  online.  Participants  used 
copy/paste  to  put  a song  text  into  a deliverable  or  to  prepare  to  do  a translation.  In  one  case,  finding  an  aria 
in  a book  was  difficult  because  it  was  not  indexed.  Numerous  resources  are  only  available  in  books; 
participants  were  comfortable  finding  and  using  these — for  longer  assignments.  For  reference  materials  that 
were  both  online  and  on  the  shelf,  preference  seemed  driven  by  familiarity  and  ease  of  access.  One 
participant  used  the  hardcopy  New  Grove  dictionaries  because  he  knew  where  they  were;  he  didn’t  know 
how  to  access  them  online. 

A lingering  question  is  whether  physical  materials  are  as  inefficient  to  retrieve  as  participants 
believe.  Searching  for  online  materials  is  not  always  fast  or  fruitful.  With  Variations,  there  is  the  potential 
for  a several  minute  wait  for  the  item  to  be  copied  to  disk  cache.  An  alternate  hypothesis  is  that  students 
don’t  want  to  get  up  and  walk  around  the  library  more  than  they  have  to.  Library  carrels  are  a sometimes 
scarce  and  never  secure  workplace;  it  may  well  be  that  students  are  concerned  about  the  security  of  their 
personal  belongings  and  also  do  not  want  to  “squander”  a scarce  resource — taking  up  a computer  carrel  but 
not  sitting  there.  Another  hypothesis  is  that  online  access  “feels”  more  efficient  even  when  it  isn’t. 

Non-Library  Online  Materials  Usage.  Participants  used  non-library  web  resources  extensively. 

For  voice  students,  recmusic.org  is  well-known  and  heavily  used.  It  contains  a very  extensive  collection  of 
art  song  texts  and  translations,  and  it  can  be  searched  and  is  indexed  by  title,  composer,  first  line,  language, 
poet,  etc.  Other  web-based  frequently  used  resources  were  Google,  aria-database.com,  and  various 
commercial  sites  selling  sheet  music  or  scores. 
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Assessing  Contextual  Design  Work  Modeling 

For  solitary  library-based  learning  activity,  it  is  unsurprising  that  the  culture  and  flow  models  are 
relatively  sparse.  Yet  all  model  types  have  provided  some  amount  of  insight  and  provoked  thinking  about 
issues  that  might  not  otherwise  have  emerged  from  unstructured  data.  For  example,  participants  did  not 
mention  feeling  limited  by  not  being  able  to  excerpt  Variations  recordings,  build  a personal  collection  of 
them,  or  share  them  with  others.  This  insight  emerged  from  our  design  thinking  based  on  the  artifact  model: 
creating  artifacts  is  a process  of  taking  raw  material  and  turning  it  into  something  changed  or  new. 

Some  data  of  interest  do  not  seem  easy  to  capture  in  the  existing  work  models  and  suggest  the 
need  to  extend  or  adapt  the  models.  The  sequence  model  does  not  have  a straightforward  way  of  indicating 
the  optionality  of  intents,  or  of  showing  the  constraints  that  may  determine  whether  a step  is  needed  or  not. 
For  instance,  if  listening  is  for  personal  study  and  not  a class  assignment,  some  steps  disappear  (locate 
assignment,  write  assignment  deliverable).  While  this  can  be  modeled  as  two  distinct  sequences  with 
separate  triggering  events,  it  would  be  more  efficient  to  introduce  a mechanism  for  indicating  optionality 
based  on  constraints  or  personal  preference. 

The  flow  model  describes  how  artifacts  and  communication  move  between  people  to  accomplish 
work.  But  how  does  the  work  of  individual  learning  occur?  None  of  the  models  represents  how  learning 
occurs  when  students  interact  with  library  material.  Insight  into  student  learning  likely  requires  a more 
detailed  contextual  inquiry  than  we  conducted,  examining  student  verbal  reports  of  decisions  and  other 
thought  processes.  Yet  had  we  gathered  all  that  information — and  we  did  gather  some  in  the  form  of 
participants’  notes  and  assignments — it  is  not  clear  where  in  these  models  we  would  represent  such  events 
as  the  dialog  of  students  with  their  materials,  the  acquisition  of  skill,  or  the  adopting  of  a perspective  or 
cognitive  framework.  To  represent  learning,  we  likely  need  additional  models  whose  constructs  are 
grounded  primarily  in  learning  theory  rather  than  in  office-based  work  practice. 

Future  Work 

This  study  produced  a rich  set  of  findings  about  where  the  existing  Variations  tool  fit  with  student 
learning  activities.  It  also  yielded  insights  into  how  students  decide  between  using  online  materials  or 
physical  materials.  Finally,  this  example  also  suggested  ways  the  Contextual  Design  models,  developed  in 
the  context  of  information  systems  design,  may  need  improvements  to  address  the  needs  of  learning 
technology  design  more  effectively. 

We  plan  to  continue  contextual  inquiry  into  digital  music  library  use,  especially  now  that 
Variations2  has  replaced  Variations.  We  will  continue  to  expand  the  variety  of  users  and  usages  we  study. 
Variations2  includes  its  own  search  interface  as  well  as  presentation  of  digitized  scores,  giving  us  new 
opportunities  to  explore  the  decision  between  using  physical  or  online  materials. 

We  also  plan  to  explore  methods  compatible  with  the  CD  modeling  approach  that  can  capture  data 
about  how  learning  happens  as  students  use  digital  libraries. 

Contextual  inquiry  requires  training  and  practice,  is  labor-intensive,  and  generates  large  amounts  of  data  to 
interpret,  model,  and  store.  For  researchers  to  be  willing  to  use  this  method,  they  will  have  to  be  convinced 
of  its  value.  A potential,  largely  unexplored  value  proposition  is  that  CD  models  may  permit  meaningful, 
efficient  comparisons  between  projects,  enabling  a core  set  of  knowledge  to  be  established.  Having  a core 
set  of  consolidated  work  models  describing  sets  of  teaching  and  learning  activities  might  be  quite  valuable 
for  instructional  technology  designers. 


Acknowledgements 

The  author  would  like  to  thank  Maggie  Swan  for  her  help  with  data  for  some  of  the  interviews,  for 
her  assistance  in  creating  the  affinity  diagram,  and  for  commenting  on  a draft  of  this  paper;  Mary  Wallace 
Davidson  for  her  draft  comments  and  help  creating  the  affinity;  and  Don  Byrd  and  Jon  Dunn  for  reviewing 
drafts  of  this  paper.  Special  thanks  are  due  to  Gary  Arvin  for  granting  access  to  his  excellent  students,  to 
whom  also — thanks.  This  material  is  based  upon  work  supported  by  the  National  Science  Foundation  under 
Grant  No.  9909068.  Any  opinions,  findings,  and  conclusions  or  recommendations  expressed  in  this 
material  are  those  of  the  author  and  do  not  necessarily  reflect  the  views  of  the  National  Science  Foundation. 

References 

Beyer,  H.,  & Holtzblatt,  K.  (1998).  Contextual  design  : defining  customer-centered  systems.  San  Francisco, 
Calif.:  Morgan  Kaufmann. 


412 


Blandford,  A.,  & Stelmaszewska,  H.  (2002).  Usability  of  musical  digital  libraries:  a multimodal  analysis.  In 
Proceedings  ofISMIR  2002  Third  International  Conference  on  Music  Information  Retrieval 
(Paris,  France,  October  2002,  pp.  231-237),  IRC  AM  - Centre  Pompidou,  Paris,  France. 

Dunn,  J.W.,  & Mayer,  C.A.  (1999)  "VARIATIONS:  A digital  music  library  system  at  Indiana  University." 
DL  '99:  Proceedings  of  the  Fourth  ACM  Conference  on  Digital  Libraries  (pp.  12-19),  Berkeley, 
CA,  August  1999.  NY:  ACM  Press. 

Holtzblatt,  K.,  & Jones,  S.  (1993).  Contextual  inquiry:  a participatory  technique  for  system  design.  In  A. 

Namioka  & D.  Schuler  (Eds.),  Participatory  design:  principles  and  practice  (pp.  177-210).  Hillsdale,  NJ: 
Lawrence  Erlbaum  Associates. 

Notess,  M.  (2004a).  Applying  contextual  design  to  educational  software  development.  In  A.-M.  Armstrong 
(Ed.),  Instructional  design  in  the  real  world:  a view  from  the  trenches.  Hershey,  PA:  Idea  Group 
Publishers. 

Notess,  M.  (2004b).  Three  looks  at  users:  a comparison  of  methods  for  studying  digital  library  use. 
Information  Research,  9(3). 

Variations.  (2005).  Variations  project  website.  Retrieved  12  October  2005,  from 
http  ://w  w w . dlib.indiana.edu/variations/ 

Variations2.  (2005).  Variations2  project  website.  Retrieved  12  October  2005,  from 
http://variations2.indiana.edu/research/ 


413 


Student  Reaction  to  a Spatial  Alternative  to  Multiple- Choice  Evaluation 


David  Richard  Moore 
Ohio  University 

Abstract 

This  paper  presents  an  innovative  software  assessment  instrument  referred  to  as  the  Spatial 
Probability  Measure  ( SPM).  The  instrument  allows  instructors  to  extend  the  multiple -choice  testing  method 
by  allowing  students  to  express  their  opinions  on  a continuum  relative  to  available  choices.  Following  a 
brief  introduction  to  the  SPM,  the  paper  presents  comments  from  students  on  their  experiences.  These 
comments  highlight  the  potential  benefits  and  limitations  of  using  the  SPM. 

The  problem  with  multiple-choice  evaluation  methods 

Although  efficient,  multiple-choice  assessment  questions  have  a number  of  inadequacies.  It  is  true 
that  carefully  designed  questions  have  the  capability  of  accurately  assessing  knowledge  in  most  learning 
domains,  even  higher-order  thinking  (Haladyna,  1994),  however,  those  indicators  may  miss  much  of  the 
tacit-knowledge  and  self-knowledge  that  can  inform  an  instructor  on  ways  to  improve  and  modify  their 
instruction.  By  forcing  students  to  select  a single  response  to  a question,  instructors  often  reward  students 
for  guessing.  Guesses,  particularly  guesses  taken  by  a student  skilled  in  test  taking,  may  skew  the  feedback 
provided  to  an  instructor. 

In  contrast,  essay  assessments  force  students  to  produce  and  construct  a response.  An  essay  may 
give  an  instructor  an  integrated  and  nuanced  view  of  a student's  knowledge  base.  Unfortunately,  the  time 
involved  in  evaluating  essays  may  be  prohibitive,  particularly  in  classes  with  large  enrollments.  Further, 
technology  has  offered  little  assistance  in  improving  the  efficiency  of  essay  evaluation.  Thus  instructors  are 
forced  to  make  choices  when  they  select  an  evaluation  technique.  The  technology  described  below  is  an 
attempt  to  provide  instructors  with  an  alternative  that  retains  the  efficiency  of  a multiple-choice  test  while 
expanding  the  range  of  data  collected. 

Introducing  the  Spatial  Probability  Measure 

The  Spatial  Probability  Measure  (SPM)  is  an  assessment  tool  that  presents  multiple-choice  style 
questions  in  a graphical  manner.  The  software  was  adapted  from  the  Admissible  Probability  Measure 
(APM)  used  in  the  PLATO  IV  computer-based  instructional  system  (Bruno,  1987;  Klinger,  1997;  Landa, 
1976).  The  SPM  was  created  in  Macromedia’s  Authorware™  and  consists  of  a graphical  triangle  display 
presented  to  the  learner  through  a web-browser.  Each  assessment  question  is  embedded  within  this  display. 
As  demonstrated  in  Figure  1,  the  learners  have  the  ability  to  place  their  cursor  anywhere  within  the  triangle, 
for  example,  in  this  demonstration  case  they  are  asked  to  give  their  opinion  on  their  favorite  color. 

Although  the  selection  of  color  is  a trivial  application,  the  instrument  can  just  as  easily  provide  learners 
with  an  opportunity  to  express  their  opinion  among  any  three  choices.  By  using  the  SPM  learners  can 
express  their  opinion  relative  to  the  given  choices,  so  no  one  alternative  must  be  selected.  For  opinion 
related  questions  such  as  illustrated  in  Figure  1 , learners  can  provide  a nuanced  response.  For  questions 
with  a single  correct  answer  the  SPM  provides  the  capability  for  expressing  a level  of  certitude  in  one’s 
answer,  an  important  indicator  of  intellectual  mastery  (Mory,  1991). 

Information  on  confidence  and  opinion  is  often  difficult  to  gather  and  collate.  It  is  hoped  that  this 
tool  can  allow  instructors  to  quickly  and  efficiently  gather  a new  set  of  data  to  inform  their  practice.  By 
measuring  the  pixel  distance  from  each  comer  to  the  point  selected,  one  may  record  how  strongly  one  feels 
about  one’s  answer.  Selecting  a position  directly  in  the  center  of  the  triangle  may  express  the  opinion  that 
each  of  the  available  alternatives  are  believed  to  be  equally  as  likely  to  be  correct  or,  if  the  question  is 
soliciting  an  opinion,  then  the  choices  are  equally  as  likely  to  be  appealing.  Selecting  a point  closer  to  one 
corner  demonstrates  that  a learner  favors  that  answer  relatively  to  the  others.  A working  example  and 
description  of  how  to  use  the  SPM  is  available  at:  http://oak.cats.ohiou.edu/~moored3/. 

Instructional  uses  of  the  Spatial  Probability  Measure 

Besides  the  opportunity  for  an  instructor  to  gather  a detailed  measure  of  a student’s  understanding, 
the  SPM  can  provide  students  with  critical  information  to  inform  their  own  metacognitive  awareness. 
Metacognitive  awareness  depends  upon  a learner’s  ability  to  accurately  and  effectively  monitor  and  their 
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level  of  knowledge  (Isaacson,  2004).  Without  this  awareness  students  may  feel  that  they  know  the  material 
better  than  they  actually  do.  The  nature  of  traditional  multiple-choice  assessments  may  contribute  to  this 
feeling  and  thus  reduce  the  perceived  need  for  additional  study.  Isaacson  (2004)  states,  “Clearly  it  is  critical 
for  students  to  know  when  they  know  and  when  they  do  not  know,  so  they  can  engage  or  disengage  in 
learning  and  adjust  their  learning  strategies  when  the  strategies  do  not  result  in  mastery  of  the  required 
material.”  The  SPM  has  been  designed  to  remove  the  effects  created  by  guessing.  The  SPM  gives 
instructors  a method  for  determining  a learner's  response  certitude  which  can  be  a valuable  asset  in 
evaluating  the  success  of  instructional  efforts. 

It  is  hypothesized  that  points  within  the  triangle  can  correlate  a student’ s response  as  being 
informed  (point  chosen  closer  to  the  “correct”  answer”,  uninformed  (point  chosen  closer  middle  of  the 
triangle)  and  misinformed  (point  chosen  closer  to  the  “wrong”  answers)  (Bruno,  1987;  Moore,  2005).  Each 
of  these  categorizations  may  be  used  to  guide  an  instructional  experience.  Conceptualizations  that  are 
classified  as  “informed”  may  require  only  a brief  visitation  and  review.  Conceptualizations  that  are 
classified  as  “uninformed”  may  provide  evidence  that  a student  has  no  basis  for  making  a choice  and 
perhaps  instruction  should  begin  at  a basic  level.  Perhaps  most  important  for  developing  instructional 
material  is  the  classification  of  “misinformed”,  in  this  case,  the  student  may  likely  have  a misconception 
that  may  interfere  with  further  learning  and  should  be  carefully  “rooted  out”  (Moore,  2005;  Zull,  2002). 

Student  Response  to  the  Spatial  Probability  Measure 

Because  the  SPM  instrument,  implemented  through  the  Internet,  is  a new  configuration  of  the 
relatively  obscure  APM,  there  are  a number  of  issues  that  are  unresolved  concerning  this  instrument’s 
effectiveness.  Students’  reaction  to  the  instrument  provides  a great  deal  of  information  as  to  its  efficacy. 
Comments  from  thirty-nine  students  enrolled  in  a graduate  level  introductory  instructional  technology 
course  were  solicited.  These  students  were  presented  with  a web-based  instructional  module  on  trochaic 
poetry  which  was  followed  by  fifteen  questions  presented  in  the  SPM  triangle  format.  The  questions 
contained  one  correct  example  of  trochaic  meter  along  with  two  non-examples,  usually  consisting  of  iambic 
or  dactylic  meter.  Instructional  examples  and  assessment  items  were  modified  from  those  used  in  a study 
by  Merrill  and  Tennyson,  (1971).  This  subject  matter  was  specifically  chosen  because  it  was  unlikely  that 
the  students  had  any  background  in  the  subject.  Students  confirmed  this  unfamiliarity  and  reported 
uniformly  low  knowledge  on  the  topic  on  a pre-survey. 

Students  were  only  asked  for  general  comments;  they  could  choose  to  comment  on  any  aspect  of 
the  experience  or  were  free  to  not  comment  at  all.  Comments  ranged  from  discussing  the  subject  matter  to 
discussing  the  SPM  instrument,  however,  only  those  comments  highlighting  the  instrument  are  categorized 
and  considered  below. 


Student  Comments 


Multiplicity  of  Answers 

• The  triangle  made  the  process  a little  better  because  I did  not  have  to  choose  one  answer,  I could 
go  between  two  or  three  answers. 

• I liked  how  the  program  was  able  to  measure  the  confidence  in  each  (answer)  and  to  show  how 
much  one  answer  was  favored  over  another. 

• I understand  the  instrument  is  used  to  show  preference  for  a certain  answer  or  more  than  one 
answer  without  expressing  certainty.  I liked  the  fact  that  if  I could  rule  out  one  answer  I could 
show  that  by  putting  my  dot  on  the  opposite  side  of  the  triangle  between  the  other  two  answers. 

I'm  not  sure  how  teachers  could  use  this  instrument  to  assess  whether  students  actually  know  the 
information,  being  able  to  rule  out  one  answer  or  being  torn  between  two  answers  doesn't  show 
that  the  student  necessarily  knows  what  he's  being  expected  to  know.  I think  that  this  could  be 
used  to  assess  how  much  students  know  about  a topic  before  it  is  studied  because  that  is  a time 
when  definite  answers  aren't  needed,  just  a general  feel  for  what  your  students  know  and  still  need 
to  learn. 

• I understood  how  I was  being  assessed  but  I do  not  understand  the  poetry  so  it  was  a little 
frustrating.  It  was  interesting  how  I could  place  the  point  between  two  answers  I would  like  to  see 
how  exactly  this  would  be  looked  at  if  all  of  my  answers  were  in  the  middle  of  two  options  instead 
of  deciding  on  one,  how  would  you  assess  that  person  with  this  method.  It  was  an  interesting 
experiment. 

• The  instrument  was  very  useful  because  I did  not  have  to  choose  just  one  answer. 
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• Very  interesting.  I like  being  able  to  not  give  a one-hundred  percent  accurate  answer. 

• I think  after  awhile  this  could  be  helpful  in  answering  questions. 

• I understood  the  instrument  real  well.  It  helps  you  sort  of  break  down  what  possibly  could  be  the 
right  without  actually  knowing. 

• I understood  the  instrument  and  I think  that  it  was  good  because  many  times  I feel  that  I can't 
decide  between  two  answers  but  I am  leaning  towards  one.  For  an  assessment  over  material  like 
this  (the  poems  and  such)  I think  that  this  is  very  good,  but  maybe  over  more  clear  answers  like 
the  president  one  at  the  beginning  it  wouldn't  be  as  successful.  Anyways  it  is  a good  idea. 

Traditional  expectations  of  multiple-choice  instrument 

• The  instrument  was  kind  of  confusing  for  me  because  I thought  I was  looking  for  the  one  answer 
that  was  correct.  The  triangle  option  was  good  with  the  colors  because  I like  both  red  and  blue  but 
red  slightly  more  then  blue.  However;  when  using  the  triangle  as  a tool  for  a question  where  there 
is  one  right  answer  I felt  like  I sometimes  wanted  to  put  it  in  the  middle  of  the  two  that  I thought  it 
might  be  or  if  I thought  only  one  was  correct  it  seemed  like  I couldn't  get  the  dot  right  on  the 
correct  point  of  the  triangle.  Depending  on  a person  I think  that  could  be  confusing  or  misleading. 

• I understood  the  instrument;  I don’t  really  like  this  more  or  less  than  multiple  choice  though.  How 
would  you  grade  something  like  this?  I am  really  indifferent  about  it. 

• I understood  it,  I think  it  could  be  valuable  to  assess  kids  in  this  way.  It  makes  it  more  fair 

Encouraging  indecisiveness 

• The  triangle  to  answer  questions  made  it  slightly  more  difficult  to  make  a definite  answer  because 
I was  tempted  to  just  choose  the  middle  when  I was  unsure. 

• I understood  the  instrument  in  that  it  allows  you  to  express  your  confidence  but  I felt  that  it  made 
me  second  guess  my  answer.  I think  if  it  was  on  a test  that  I understood  the  material  previously 
that  it  may  have  some  counterproductive  effects.  I may  not  be  as  confident  in  my  answer  and  play 
it  safe  by  putting  the  dot  more  towards  the  middle.  Interesting  though. 

• I was  less  reluctant  to  pick  an  actual  answer.  Knowing  that  the  option  was  available  for  me  to  be 
indecisive,  I think  that  I was  less  sure  then  I would  have  been  if  I could  have  only  chosen  one 
answer. 

Interesting  behavior 

• I noticed  that  my  favorite  color  on  the  triangle  corresponded  with  the  majority  of  my  answers  on 
the  poetry  section. 

• I liked  it,  but  I think  it  would  be  hard  to  use  it  for  a written  test.  It  almost  has  to  be  used  with  a 
computer  test. 

• I like  the  instrument  but  it  might  be  better  tested  in  a subject  more  readily  understood. 

Clarity  of  the  instrument 

• Marking  the  spot  in  the  triangle  was  understandable 

• I understood  the  instrument  and  think  it's  a really  novel  idea. 

• I feel  like  when  I was  given  the  choice  of  3 and  I could  pick  any  space  between  them. 

Deficiency  of  the  software  trial 

• The  ability  to  change  your  mind  before  submitting  might  be  handy. 

• The  assignment  was  easy  to  understand;  but  time  consuming.  As  a result;  I lost  interest  by  the  end. 

• Too  many  examples  of  using  the  triangle.  I did  not  even  look  at  the  last  five  or  six,  I just  marked 

the  middle  because  I didn't  know. 

Discussion  and  Classroom  Implications 

Many  of  these  students  favored  not  being  forced  to  select  a particular  answer.  These  positive 

comments  may  highlight  the  fact  that  a multiple-choice  test  is  a particularly  artificial  type  of  assessment. 

Norman  (1988)  points  out  that  the  environment  in  which  we  apply  knowledge  provides  an  information-rich 

context  that  we  react  to  and  interact  with;  without  these  cues  we  often  are  less  certain  about  our  knowledge 
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base  than  is  commonly  accepted.  The  students’  comments  may  recognize  the  artificial  nature  of  discrete 
responses. 

The  SPM  instrument  was  new  to  the  students  in  this  demonstration  and  it  is  likely  to  be  new  to  most 
students  who  encounter  it.  Like  other  instruments,  such  as  Classroom  Assessment  Techniques  (Angelo  & 
Cross,  1993),  learners  require  some  degree  of  introduction,  explanation,  and  perhaps  training  and 
experience  to  grasp  the  meaning  and  purpose  of  the  instrument.  While  students  are  becoming  familiar  with 
an  instrument  such  as  the  SPM,  it  may  be  best  used  informally  and  perhaps  with  the  results  being  reviewed 
as  an  in-class  group  exercise.  A productive  implementation  could  be  a classroom  outfitted  with  a digital 
projector  that  displays  the  response  triangle  and  a group  of  students’  responses.  Students  can  be  asked  to 
explain  their  selections  and  their  assumptions  regarding  their  responses.  This  approach  may  be  able  to 
clarify  the  students’  intentions. 

The  comments  revealed  a certain  uncertainty  among  the  respondents  concerning  the  SPM.  It  is 
likely  that  students  have  a preconception  of  what  a legitimate  assessment  instrument  looks  like.  Many 
students  appeared  not  to  understand  the  notion  of  a continuum  of  answers  compared  to  the  discrete 
responses  required  in  multiple-choice  tests.  With  this  in  mind,  many  students  discussed  a potential 
downside  of  the  SPM.  Many  students  expressed  the  opinion  that  the  nature  of  the  SPM  encouraged 
indecisiveness;  that  it  allowed  them  not  to  think  carefully  about  the  question.  It  remains  to  be  seen  whether 
this  perception  would  endure  once  their  experience  with  the  instrument  increases. 

Based  on  these  comments,  the  SPM  instrument  seems  to  hold  he  potential  for  further 
experimentation  and  investigation.  The  SPM  has  the  potential  to  efficiently  gather  data  that  highlights  a 
student's  comprehension  level  in  a unique  way.  Instructors  and  researchers  are  encouraged  to  test  and  try 
out  the  instrument  in  their  own  classrooms.  Additional  technical  details,  demonstration,  and  question 
templates  are  available  at  http://oak.cats.ohiou.edu/~moored3/SPM. 
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Teaching  and  Learning  for  Performance: 

A Motivational  Instructional  Design  Model  for  Improving  Student 
Academic  Achievement  in  a Hybrid  Learning  Environment 


Mrs.  E.  T.  Okon 
University  of  Houston 

Introduction 

The  theory  (Teaching  and  Learning  for  Performance  - TLF)  that  has  been  formulated  in  this  paper 
focuses  on  expositing  the  practical  ways  in  which  an  instructional  design  model  can  be  used  to  ascertain  the 
achievement  of  increasing  performance  in  instruction  and  learning  in  the  domain  of  natural  languages  and 
other  content-rich  courses.  College-bound  and  post-secondary  learners  are  the  primary  focus  of  this  theory. 

Since  the  combination  of  perspectives  or  philosophies  is  an  important  aspect  of  the  new  paradigm 
of  instructional  theories,  and  as  David  Jonassen  states,  some  of  the  best  environments  use  combinations  of 
methods  (Reigeluth,  1999,  p.  217),  this  theory’s  framework  straddles  the  “broadly  constructivist”  theory  of 
David  Perkins  and  Chris  Unger,  and  the  cognitivist  theory  of  Jerome  Bruner. 

This  paper  elaborates  on  the  following  topics: 

• Definition  of  Instructional  Design  Theory 

• The  theoretical  framework  and  precursors 

• Design  components 

• Goals  and  preconditions 

• Values 

Team  work  and  Motivation  of  learners 

Learning  in  a mode  that  fosters  unrestrained  academic  interaction  and  creativity  learners 
Effective  and  efficient  acquisition,  construction  and  transfer  of  knowledge 
Cultural  Diversity 
Preparation  for  Lifelong  Learning 

• Learners 

• Content 

• Environment/Medium  of  Delivery 

• Teaching  versus  Learning  Styles 

• Instructional  methods 

• Assessment/Evaluation  methods 


Conclusion 

• Major  contribution  (s)  of  the  theory  in  the  domain  of  instructional  design  theories  and  models 

• The  future  of  instructional  design  theory 

Definition  of  Instructional  Design  Theory 

As  stated  by  Charles  M.  Reigeluth,  instructional  design  theory  is  a theory  that  provides  clear 
guidance  on  ways  to  effectively  help  people  learn  and  develop  (Reigeluth,  1999).  This  theory  is  concerned 
with  ways  that  learners  may  be  helped  to  obtain  higher  rates  of  success  at  the  individual  as  well  as  at  the 
group  level. 


Theoretical  Framework 

In  instructional  design,  the  subject  (course)  determines  the  theoretical  tilt  of  the  design,  (cf. 
Instructional  Design  and  Learning  Theory 

http://www.usask.ca/education/coursework/802papers/mergel/brenda.htm#The%20Basics%20of%20Constr 
uctivism  Additionally,  task-based  language  teaching  (which  forms  part  of  the  language  lesson  content  e.g. 
how  to  write  a business  letter)  is  concerned  with  communicative  and  cognitive  processes”  ( Approaches  to 
Foreign  Language  Syllabus  Design  http://www.ericdigests.org/pre-928/design.htm.) 
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As  has  been  said,  the  focus  of  this  theory  is  on  language  teaching  and  learning,  and  although 
Bruner  (1983)  has  to  do  with  language  acquisition  by  young  children  http://tip.psychology.org/bruner.html, 
his  instructional  theory  principles  can  be  adapted  to  the  instruction  of  older  children. 

Since  therefore  TLF  theory  hinges  upon  a principle  of  Bruner’ s instructional  theory  of  cognitive 
development  ( Instruction  must  be  structured  so  that  it  can  be  easily  grasped  by  the  student  ( spiral 
organization ),  and  Howard  Gardner’s  theory  of  Multiple  Intelligences  which  applies  to  all  ages 
(http://tip.psychology.org/gardner.html),  design  components  appropriate  for  language  instruction  and 
learning  have  been  identified  in  the  process  of  developing  this  instructional  design  theory  that  is  also 
derived  from  David  Perkins’  and  Chris  Unger’s  Teaching  and  Learning  for  Understanding  (TfU)  theory 
(. Knowledge  construction,  that  is,  understanding,  requires  performance)  which  assumes  a broadly 
constructivist  perspective  to  instructional  design  (Reigeluth,  1999,  pp.  91-  114) 

Design  Components 

Robert  Gagne  originated  systematic  instructional  design,  analyzing  learners  and  course  goals  to 
make  objectives,  sequence  instructional  experiences,  set  the  medium  of  instruction,  and  assess  student 
performance  and  the  course  (re:  Judith  V.  Boettcher,  Ph.  D. 

http://vccslitonline.cc.va.us/usingweb/bckgrnd.htm).  This  systematic  design  of  instruction  has  the  primary 
goal  of  helping  students  learn. 

Goals  and  Preconditions 

The  goal  and  precondition  of  this  theory  are  to  achieve  the  impartation  of  knowledge  and  to  foster 
learning  in  a safe,  caring,  supportive  and  non-threatening  but  work-oriented  and  performance-provoking 
face-to-face  and  online  environment. 


Values 

Team  work  and  Motivation  of  Learners 

Motivation  through  the  use  of  collaborative  learning  segments,  especially  in  the  case  of  online 
portions  of  a course,  is  a possible  way  of  reducing  the  attrition  rates. 

The  Geese  Illustration:  The  quality  of  support  in  teamwork  and  communication  demonstrated  by 
geese  is  worth  emulating.  This  lesson  from  nature  can  be  applied  to  support,  and  communication  (transfer) 
of  knowledge  among  learners.  According  to  a scientific  study,  geese  fly  in  a V formation  (a  principle 
employed  by  the  Tour  de  France  bicyclists)  for  reasons  of  efficiency.  “It's  called  "drafting"  — the  geese 
use  the  air  currents  created  by  the  birds  in  front  of  them  to  fly  more  efficiently.  The  aerodynamic  V shape 
reduces  the  air  resistance,  which  lets  the  geese  fly  up  to  70  percent  farther  than  if  they  were  flying  alone. 
And  since  the  bird  at  the  front  of  the  "V"  encounters  the  most  wind  resistance,  they  all  take  turns  at  that 
position.”  (Ask  Earl , http://vahooligans.yahoo.com/content/ask  earl/page?d=20020808).  “Flying  in  a "V" 
also  has  social  advantages:  it  allows  birds  to  communicate  with  each  other  while  on  the  wing.”  (Bird  flight 
explained , http://news.bbc.co.Uk/l/hi/sci/tech/160825 1 .stm) 

Responsibility  (assignment  of  tasks  that  require  a learner  to  be  responsible  to  fellow  learners,  e.g. 
preparation  of  a presentation  on  a topic),  interdependence  and  respect  of  others,  and  the  boosting  of  each 
other’ s self  esteem  (while  telling  each  other  the  truth  about  their  performance  when  called  upon  to  make 
peer  evaluations)  are  ways  of  motivating  learners  to  stay  focused  on  learning. 

Display,  by  the  instructor,  of  all  work  accomplished  by  students,  without  showing  the  individual 
students’  names,  on  a web  page  accessible  to  the  public  (parents,  students,  et  al)  with  comments  regarding 
what  features  made  which  work  outstanding,  and  with  the  explanation  that  it  is  a method  of  encouraging 
learners  to  aim  higher  in  their  learning  and  performance. 

Mid-Term  Awards  and  End-of-Quarter  Awards,  etc.  are  additional  extrinsic  motivational  factors. 

Further,  responsibility  and  boosting  of  self-esteem  could  be  achieved  through  cooperative  learning 
http://www.iigsaw.org/overview.htm.). 

Learning  in  a mode  that  fosters  unrestrained  academic  interaction  and  creativity  among  learners 

When  learners  feel  safe  to  unveil  their  inadequacies  of  knowledge  and  ask  “stupid”  questions,  they 
will  learn  from  the  correct  answer  offered  by  the  instructor  or  by  peers.  The  generation  of  knowledge  could 
result  under  such  circumstances. 
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Effective  and  efficient  acquisition,  construction  and  transfer  of  knowledge 

The  creation  of  a relatively  relaxed  learning  atmosphere  could  foster  the  acquisition,  construction 
and  transfer  of  knowledge.  This  situation  could  be  created  through  the  removal  of  the  threat  of  failure  or 
the  threat  of  the  less-knowledgeable  gaining  grounds  and  out-performing  the  brilliant  student  - there  is  no 
harm  if  all  students’  performance  is  at  a peak.  However,  healthy  competition  should  be  encouraged. 

Cultural  Diversity 

With  various  parts  of  the  world  becoming  multicultural  through  globalization,  educators  should 
recognize  good  aspects  of  “foreign”  cultures.  The  literature  texts  could  be  enriched  by  drawing  from  this 
store.  In  H.  Gardner’s  Multiple  Intelligences,  “each  culture  tends  to  emphasize  particular  intelligences” 
http://tip.psvchology.org/gardner.html  . Hence,  the  cultural  context  is  a value  in  instructional  design 
theory. 


Preparation  for  Lifelong  Learning 

Students  should  be  enabled  to  acquire  the  discipline  of  life-long  learning  which  is  facilitated  by 
the  Internet  with  its  often  overwhelmingly  rich  store  of  resources.  Dewey,  for  example,  claimed  that 
education  should  enable  individuals  to  continue  growing  all  their  lives  intellectually,  artistically  and 
socially  http ://w ww.siu.edu/~de weyctr/a  short  annotated  reading  list.htm 

Learners 

The  learners  that  are  addressed  in  this  theory  are  college-bound  and  higher  education  students. 

Content 

As  has  already  been  stated,  the  content  for  the  design  theory  is  natural  languages  and  other 
content-rich  courses.  Conscious  effort  should  be  made  to  present  instructional  material  in  a form  that 
facilitates  storage  and  retrieval  by  learners. 

Environment/Medium  of  Delivery 

The  courses  are  hybrid,  that  is,  part  face-to-face,  and  part  online.  The  learning  environment  and 
the  testing  environment  should  be  very  similar  so  as  to  aid  recall  of  subject-matter  areas  that  are  tested  (re: 

) 

Teaching  versus  Learning  Styles 

It  is  imperative  that  in  the  new  instructional  paradigm  where  instruction  is  learner-oriented 
(Reigeluth,  1999),  teaching  styles  be  predicated  upon  learning  styles  and  on  the  learning  goals. 

Instructional  methods 

Learning  styles-dependent,  the  instructional  methods  should  be  multivariate  in  nature  in  order  to 
allow  for  a fair  coverage  of  multiple  intelligences  (linguistic,  musical,  logical,  spatial,  kinesthetic, 
intrapersonal,  and  interpersonal)  (Reigeluth,  1999).  “A  good  teacher  will  incorporate  activities  in  the 
classroom  to  facilitate  learning  for  each  of  the  seven  intelligences.”  A Collection  of  Theories  and  Theorists: 
an  aid  to  the  student  of  educational  theory.  http://www.theshop.net/aboatman/edtheory.htm#Hunter.  This 
makes  it  imperative  that  the  instructor  know  his/her  students. 

Further,  the  graduation  of  tasks  from  simple  to  complex  makes  for  the  building  of  new  knowledge 
on  previous  knowledge  and  allows  for  effectiveness  and  efficiency  in  teaching  and  learning.  But  we  realize 
that  sequencing  may  not  always  prove  to  be  the  best  trail  to  follow  as  the  students  are  guided  through  the 
twists  and  turns  of  the  knowledge  labyrinth.  Spiral  organization  of  the  curriculum  helps  students 
continually  build  upon  prior  knowledge  (cf.  Constructivist  Theory  (J.  Bruner)  Overview 
http://tip.psvchology.org/bruner.html 

Even  though  Bandura  states  that  we  can  leam  novel  behavior  without  any  practice  or  direct 
reinforcement  (Learning  by  observation),  this  theory  posits  that  there  are  cases  where  practice  (e.g. 
performance  of  a task  with  acquired  skills  (Reigeluth,  1999)  aids  memory  and  learning  and  this 
demonstration  of  knowledge  through  practice  enables  the  instructor,  the  learner  and  the  public  to  fully 
ascertain  that  the  learner  has  understood  the  lesson.  Thus,  since,  as  Mayer  states,  “the  instructional 
designer’ s role  is  to  create  environments  in  which  the  learner  interacts  meaningfully  with  academic 
material,  including  fostering  the  learner’s  processes  of  selecting,  organizing,  and  integrating  information” 
(Reigeluth,  C.  Instructional-Design  Theories  and  Models,  1999,  p.  144),  there  has  been  a conscious  effort 
made  to  ensure  the  proper  functioning  of  this  design  for  the  purpose  of  facilitating  and  enhancing  the 
learning  process  in  a way  that  could  spike  learners’  performance. 

The  responsibility,  punishment  and  reward  systems  that  have  been  put  in  place  e.g.  doing 
homework,  filling  out  a progress  chart  and  making  graphs  from  it  to  see  their  learning  curve  periodically 
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(with  the  help  of  the  instructor),  participation  in  team  work,  and  Drill-and-Practice  exercises  accompanied 
by  the  instructor’s  open  approval  of  students’  good  performance,  disapproval  or  ignoring  of  student 
misbehavior  if  any,  and  disapproval  and  genuine  show  of  concern  and  of  disappointment  at  student  poor 
performance  if  any,  are  such  as  are  intended  to  encourage  learning. 

Retention  of  students’  attention  (e.g.  by: 

1 . abruptly  changing  the  tone  of  voice  or 

2.  periodically  stopping  within  the  lesson  to  ask  a question  that  requires  a feedback  from  students 
to  keep  on  course  any  whose  thoughts  may  have  wandered  off)  a key  to  fostering  learning. 

The  following  activities  can  be  included  to  make  learning  meaningful  and  interesting. 

Short  story  (used  as  an  advance  organizer):  Told  by  teacher  or  solicited  from  students,  a short  story, 
aptly  used,  can  clarify  the  lesson  (Reigeluth,  1999,  p.  155). 

Questions  (including  within-the-lesson  attention  retainers),  Quiz  or  Puzzle:  These  can  serve  as  lesson 
reinforcers. 

Video  Shows:  Although  the  combination  of  words  with  other  words  can  achieve  much  more  than  what 
images  and  acts  can  achieve,  combining  words  with  images  and/or  acts  proves  to  be  an  even  more  powerful 
and  productive  strategy  (cf.  Bruner,  1964,  p.2,  cited  in  Driscoll,  2005,  p.  229). 

Individual  attention  given  to  students:  This  could  be  the  answer  to  many  students’  learning  needs. 
Reference  to  prior  taught  content:  The  spiral  nature  of  the  lessons  make  for  continuous  retention  of 
lesson  content. 

Provision  of  Online  Tutorials/Tutorial  Software  (CDs,  DVDs,  etc.):  They  serve  as  untiring  instructors. 
Online  Quiz/Drill-and-Practice  Software:  This  is  very  useful  for  lesson  reinforcement. 

Music:  For  musical  learners,  the  occasional  putting  of  salient  lesson  points  to  music  will  help  some 
learners  in  retention  of  the  lesson  (cf.  Gardner’s  Multiple  Intelligences,  in  Reigeluth,  1999) 

Poems:  The  use  of  poems  could  also  foster  learning  for  the  poetically-inclined. 

The  “Listen- While- You-Read”  Program  with  streaming  audio  or  downloadable  audio  lesson 
segments  (cf.  Audiobookshelf:  http://www.audiobookshelf.com/eom  feb04.html ).  This  program  has  been 
found  to  be  of  immense  help  to  students,  helping  them  to  read  more  and  probably  faster,  in  the  case  of  slow 
readers.  Audio  design  is  needed  for  education  to  fully  profit  from  its  use  in  formatting  educational  content. 
Knowledge  is  needed  regarding  cognitive  load  and  effective  multimodal  strategies  (Sweller,  1988;  Sweller, 
Merrienboer,  & Paas,  1998;  Mayer,  2001;  cited  in  Barron,  2004).  For  example,  embedded  audio  should  be 
manipulable  in  the  sense  that  learners  should  be  able  to  adjust  the  volume  or  turn  it  off  as  needed. 
Drawings,  maps,  pictures,  cartoons,  illustrations,  animations  (Reigeluth  1999,  page  155). 

These  bring  the  lesson  to  life,  reinforce  it  and  improve  understanding,  thus  helping  learners  integrate 
knowledge.  However,  they  and  other  graphics  should  not  be  redundant  so  as  not  to  interfere  with  the 
retention  of  more  relevant  information  (cf.  Reigeluth.  1999,  page  153) 

According  to  Bandura,  we  store  events  in  two  ways-through  visual  images  and  through  verbal  codes. 
Worked-out  Examples  and  Elaborative  questions:  It  has  been  proven  that,  students  who  received 
elaborative  questions  performed  better  in  problem-solving  and  remembering  lesson  content  (Reigeluth, 
1999,  p.  155). 

Color  coding  of  text:  This  serves  to  enhance  the  learning  process  (Erlauer,  2003;  Hall  & Sidio-Hall, 
1994). 

Short-term  Job  Placements  (Internship)  also  help  to  reinforce  educational  content  (cf.  Learning  by 
Doing  theory  of  Instructional  Design  in  Reigeluth,  1999,  pp. 161 -181). 

Discussion  among  students  (face-to-face  and  online):  Communication  among  students  as  well  as  between 
students  and  the  instructor  helps  students  get  the  lesson  straight  if  they  failed  to  do  so  when  the  lesson  was 
first  taught. 

Internet  search  (it  is  hoped  that  the  school’s  filters  are  effective): 

Directed  internet  search  where  students  are  given  precise  instructions  regarding  the  search  (objectives  are 
clear),  instead  of  being  let  loose  on  a wild  goose  chase  in  the  internet  resource  jungle,  (cf.  Jerome  Bruner’s 
learning  theory,  in  http://vccslitonline.cc.va.us/usingweb/bckgrnd.htm)  is  an  instructional  method  that  can 
yield  good  fruits. 

Presentation  by  instructor:  Presentation  by  the  use  of  software  such  as  PowerPoint  adds  interest  to 
learning.  The  use  of  powerful  summaries  by  instructors  and  instructional  designers  in  course  materials  will 
foster  student  learning. 


421 


Team  Teaching:  Most  students  who  participated  in  a study  believed  eventually  that  team  teaching  fostered 
their  understanding  of  the  learning  content  (Hwang,  Hernandez,  & Vrongistions,  2002;  Wenger  & 

Hornyak,  1999,  cited  in  Hwang,  Hernandez,  & Vrongistions,  2002). 

Preparation  and  Delivery  of  Presentations  by  students:  Students’  performance  can  be  openly  and  fairly 
confirmed. 

Preparation  and  Production  by  students  of  Movie  Project  (under  instructor’s  or  a Subject-Matter 
Expert’s  supervision) 

Teaching  of  Specialized  topics  by  a Subject-Matter  Expert:  Engaging  a subject  matter  expert  (SME)  to 
teach  a specialized  topic  is  a way  of  helping  learners  learn. 

Use  of  repetition  and  emphasis  (up  to  six  different  times  if  necessary)  and  summary  cf.  Mayer  (1993, 
cited  in  Reigeluth,  1999,  page  152) 

Use  of  analogies,  synonyms  and  examples:  These  help  to  make  the  lesson  easier  to  remember  (Reigeluth, 
1999). 

“Concretization”  of  abstract  information  through  e.g.  “picture  words”  and  symbols  in  explanation 
“Accelerated  Learning”  segments  through  the  use  of 

1.  Faster  audio  clips  cf.  audiotape  use  in  Audiobookshelf: 
http://www.audiobookshelf.com/eom  feb04.html 

2.  Accelerated  Writing  (making  students  practice  to  write  faster  so  as  to  liberate  time  for  the 
completion  of  more  learning  and  the  demonstration  of  understanding  through  performance,  thereby 
probably  increasing  learning  outcomes). 

Flexibility 

Any  of  the  above  can  serve  as  an  entry  point  into  the  lesson  (Reigeluth,  1999,  pp.  8,  25) 

Assessment/Evaluation 

The  use  of  multiple  assessment  methods  helps  to  incorporate  all  learning  styles,  ascertaining  the 
comprehensiveness  of  the  assessment/evaluation: 

1.  Class  work 

2.  Attendance  and  Participation  in  Class  Discussions  (face-to-face  and  online)  (to  test  critical 
thinking  skills) 

3.  Homework  (to  assess  autonomy  levels) 

4.  Team  work  (to  test  collaboration  skills) 

5.  Project/Presentation  (to  test  creativity,  organization,  evaluation,  judgment) 

6.  Short-term  Job  Placement  (to  test  performance  in  application  environment.  According  to  Dewey, 
what  happens  in  the  classroom  should  be  an  integral  part  of  the  activities  of  the  wider  community 
http://www.siu.edU/~deweyctr/a  short  annotated  reading  list.htm ) 

7.  Student  portfolio  of  reflection  and  products  in  course 

8.  Traditional  method:  Examination  (multiple-choice,  fill-in-the-blanks,  open-ended  questions) 
Grading  rubrics  (should  be  made  public  to  students,  administrators,  parents)  to  provide  evidence  of 
fairness  and  quality.  Further,  the  rubric  makes  room  for  contingencies  (unforeseen  positive  performance 
“surprises”). 


Conclusion 

Major  Contributions 

In  the  face  of  positive  global  development  strides  that  have  been  taken  in  various  areas  especially  in 
science  and  technology,  efforts  are  being  made  in  education  to  prevent  the  formation  of  a yawning  gap 
between  the  “classroom”  and  “real  life”  in  the  society.  This  leads  to  the  making  of  contributions  in  the 
domain  of  instructional  design  theory  by  instructional  designers.  Hence,  following  are  this  paper’s 
contributions  to  instructional  design  theory: 

“Accelerated  Reading  and  Writing”  could  spawn  novel  levels  of  academic  performance 

Creation  of  an  environment  to  foster  the  sharing  and  hence  the  generation  of  knowledge  by  learners 

due  to  the  absence  of  the  threat  of  the  less  knowledgeable  outperforming  those  who  freely  shared 

knowledge. 

Formative  evaluation  of  the  theory  will  be  carried  out  and  the  external  validity  of  the  research  will  be 
assessed. 

The  Future  of  Instruction,  Learning,  Instructional  Design  and  Assessment 

The  enlargement  and  enrichment  of  the  curriculum,  a feature  of  the  new  paradigm  of  instruction,  has 
implications  for  instructional  design  theories.  Due  to  the  growing  body  of  knowledge  about  instruction 
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(Reigeluth,  1999)  theorists  have  to  rethink  and  bring  forth  theories  that  will  be  suited  to  the  meaningful 
customization  of  instruction,  for  example.  This  development  equally  has  major  implications  for 
professional  development  - instructors  may  have  to  relearn  how  to  teach  effectively  and  efficiently  in  the 
future  classroom.  The  introduction  of  assessment  options  in  training  and  higher  education  looks  feasible 
already.  Educators  will  have  to  set  new  performance  standards  and  further  adapt  instruction  to  the  needs  of 
individual  learners. 

Figure  1:  Student  Progress  Information 


PERFORMANCE  vs  TIME 


TIM  E 


Each  learner’ s starting  point  in  a course  should  be  noted  in  the  chart  in  order  to  determine  individual 
progress  rate.  The  effect  of  differentiating  instruction  is  variation  in  course  completion  time  among 
students. 

Figure  2:  Peak  Performance  Expectations 


Learners  could  also  be  motivated  through  the  graphical  representation  of  their  performance  in 
relation  to  the  course  objectives  and  peak  performance  expectations  at  various  stages  in  the  course. 
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Figure  3:  Instructional  Process  Cycle 


Course 

content 


Preliminary 


content  summary 


Instruction  begins  with  a preliminary  content  summary,  leading  to  entire  course  content  presentation 
in  units,  and  then  ending  in  a summary  of  crucial  course  content. 

Figure  4:  Motivational  Instructional  Design  Model  (Derived  from  Dick,  Carey,  & Carey,  2005  and 
Motivated  Strategies  for  Learning  Questionnaire,  MSLQ): 

Analysis  of  Learner  and  Instructor  Characteristics  and  their  goals  for  the  course 

Environmental  design 


Course  objectives  and  Expectations Content  design  (for  differentiated  instruction) Design 

of  Motivated  strategies  for  instruction  and  learning — Assessment  instruments  design 

Evaluations  (formative  and  summative) 
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Figure  5:  Teaching  and  Learning  for  Performance  (TLP)  Design  Model  for  Content 
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Instructional  Design  Theory  for  Integrating  Learner- Centered 
Perspectives  into  Entirely  Web-Based  Courses  in  Higher  Education 
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Abstract 

This  study  developed  an  instructional  design  theory  to  provide  theoretically  sound,  practical 
guidelines  that  incorporate  learner-centered  and  collaborative  approaches  into  entirely  Web-based  ESL 
pragmatic  courses.  The  formative  research  methodology  was  used  to  create  and  refine  the  design  theory. 
The  initial  theory  was  developed  inductively  based  on  various  resources.  For  the  theory  refinement,  the 
prototype  model  was  formatively  reviewed  by  experts  invited  from  related  fields.  The  refined  design  theory 
identifies  instructional  values  and  instructional  conditions  and  suggests  instructional  methods. 

Web-Based  Instruction  in  Higher  Education 

Internet  technology  continues  to  accelerate  the  growth  of  distance  education  in  higher  education 
(Molenda  & Harris,  2002;  NEA,  2000).  According  to  a survey  of  the  National  Education  Association 
(2000),  90  percent  of  higher  institutions  offered  distance  courses  and  programs  and  the  Web  is  the  most 
used  delivery  mode  for  distance  courses  (44%).  Advocates  note  that  the  Internet  makes  education  more 
flexible  and  accessible  for  traditional  and  non-traditional  learners  (Web-based  Education  Commission, 
2000)  and  its  technology-rich  environments  create  dynamic  instructional  contexts  that  are  different  from 
traditional  classrooms  (Kahn,  2001).  Research  has  also  shown  that  the  Internet  has  the  potential  to  promote 
interactivity  (Polin,  2004),  foster  higher-order  thinking  skills  (Bonk  & Cummings,  1998),  and  promote 
social  interaction  among  participants  (Oren,  Mioduser,  & Nachmias,  2002). 

However,  teaching  online  is  a challenging  task  to  many  faculty  accustomed  to  traditional  teacher- 
led  approaches  (Ko  & Rossen,  2004;  Palloff  & Pratt,  1999).  Many  qualified  faculty  hesitate  to  participate  in 
it  (Betts,  1998;  Schifter,  2000).  When  asked  about  barriers  inhibiting  their  participation  in  distance 
education,  faculty  rank  the  lack  of  guidance  for  pedagogy  in  designing  instruction  as  one  of  the  barriers  for 
their  willingness  (Park,  2004).  Furthermore,  there  are  some  misconceptions  about  teaching  online.  Some 
faculty  and  administrators  are  misinformed  that  there  is  no  difference  in  practice  between  face-to-face 
courses  and  Web-based  courses  (Hill,  1997).  This  misinformation  leads  them  to  transfer  the  lecture-based 
teaching  pedagogies  to  teaching  Web-based  courses  (Knowlton,  2000).  Often,  it  is  observed  that  intuition 
or  tacit  knowledge  is  used  to  design  online  courses  without  considering  critical  components  of  Web-based 
instruction  (e.g.,  course  content,  pedagogical  value  of  the  instructor,  the  characteristics  of  learners,  and 
learning  context)  (Garrison,  2000;  Taylor,  2002). 

Another  concern  is  that  for  the  design  of  Web  instruction,  instructors  tend  to  choose  software 
packages  based  on  their  low  cost  or  availability  (Cyrs,  1997;  Palloff  & Pratt,  1999).  Bonk  and  Dennen 
(2003)  note  that  most  e-learning  tools  provide  “templates  and  guidelines  for  warehousing  students  and 
providing  static  course  material”  (p.  332).  Duffy  and  Jonassen  (1992)  attribute  the  static  design  to 
objectivist  principles  on  which  current  instruction  is  typically  based.  Such  design  principles  cannot  utilize 
the  potential  the  Web  affords  and  generate  low  interactions,  boring  classes,  and  arbitrary  schedules  that 
result  in  decreasing  students’  motivation  and  increasing  the  dropout  rate  of  the  course  (Update,  2002). 

To  overcome  the  problems,  researchers  propose  new  pedagogical  approaches  for  Web  instruction 
design  that  support  major  skills,  such  as  collaborative  knowledge  construction,  learner  initiative,  and 
reflection  (Oliver  & McLoughlin,  1999).  Reigeluth  (1999;  2003)  draws  attention  to  the  need  for  the  new 
paradigm  of  instruction  to  meet  the  demands  of  a rapidly  changing  society  (the  information  age).  He  calls 
for  new  instructional  design  guidelines  to  foster  abilities,  such  as  creative,  self-regulated,  and  collaborative 
competences  with  critical  thinking  skills  and  problem  solving  skills. 

Web-Based  Instruction  in  the  ESL  Field 

Since  the  Internet  was  introduced  to  the  English  as  a Second  Language  (ESL)  field  as  a new 
instructional  medium  in  the  1990s,  teaching  and  learning  - that  mainly  took  place  within  a classroom 
alienated  from  the  actual  use  of  native  speakers  of  English  - have  changed.  Web-based  instruction  has 
supported  instructors  in  creating  authentic  and  collaborative  learning  contexts  for  more  dynamic  language 
learning  (Kramsch,  A’ Ness,  & Lam,  2000).  Long  and  Richards  (2000)  claim,  “Because  computers  have 
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opened  up  new  opportunities  for  communication  between  both  learners  and  teachers  and  among  second 
language  users  themselves,  many  language  teachers  see  great  potential  in  computer  mediated  teaching  and 
learning”  (p.  ix).  Learners  also  take  advantage  of  the  Internet  in  ways  that  were  not  possible  before.  They 
can  have  access  to  myriad  resources  related  to  course  content  from  remote  locations  around  the  world. 
Web-based  instruction  as  an  open  system  allows  learners  with  more  room  for  consulting  outside  experts. 
Classroom  activities  are  no  longer  limited  to  interaction  with  peer  students  in  the  class  (Kern  & 

Warschauer,  2000).  Kern  and  Warschauer  claim, 

If  our  goal  is  to  help  students  enter  into  new  authentic  discourse 
communities,  and  if  those  discourse  communities  are  increasing  located 
on-line,  then  it  seems  appropriate  to  incorporate  on-line  activities  for  their 
social  utility  as  well  as  for  their  perceived  particular  pedagogical  value  (p.  13). 

Citing  Kramsch  et  al  (2000),  Reynard  (2003)  emphasizes,  “Use  of  an  electronic  medium,  such  as  the 
computer,  to  teach  language  has  developed  a greater  need  and  expectation  of  authenticity  and  authoring  on 
the  part  of  the  learner”  (p.  7).  It  is  important  to  note  that  authenticity,  authoring,  and  active  learner 
participation  in  Web-based  language  learning  environments  cannot  be  achieved  by  the 
use  of  traditional  teacher-led  approaches,  which  are  viewed  as  problematic  since  teacher 
dominance,  passive  students,  lack  of  authentic  inputs  and  interactions,  and  unnatural 
communications  cannot  foster  students’  use  of  language  in  a meaningful  way  (Kasper, 

1997;  Meskill  & Ranglova,  2000).  Levy  (1990)  claims,  “Our  teaching  philosophy,  method, 
or  approach  needs  to  be  broadened  to  encompass  new  technologies  and  the 
interrelationship  between  language  teaching  and  computing  needs  to  be  carefully 
explored”  (p.  5). 


Needs  of  Instructional  Design  Theory 

However,  despite  the  expansion  of  online  courses  and  the  growing  need  for  instructional 
frameworks  and  guidelines,  a dearth  of  knowledge  about  online  pedagogy  and  few  design  theories  for  an 
entirely  Web-based  ESL  course  have  been  developed.  Egbert,  Chao,  and  Hanson-Smith  (1999)  note, 
“Although  most  CALL22  texts  are  themselves  technology  driven,  that  is,  organized  around  the  types  of 
activities  computers  are  able  to  do,  few  have  spoken  to  the  need  for  a specific  framework  based  on  the  best 
of  second  language  acquisition  (SLA)  research.”  (p.  ix).  Furthermore,  it  is  observed  that  some  guidance  for 
Web-based  language  learning  is  still  descriptive  or  overlooks  some  critical  elements  that  have  been 
consistently  emphasized  in  the  education  and  WBI  fields. 

To  take  fully  advantage  of  Internet  technology,  researchers  should  pay  attention  to  a new 
paradigm  of  instruction  for  online  language  teaching  and  learning  and  address  critical  issues  to  answer  the 
question  of  how  to  integrate  Web  technologies  most  effectively  and  most  efficiently  into  Web-based 
language  learning  courses  (Kern  & Warchauer,  2000).  Therefore,  it  is  important  to  create  an  instructional 
design  theory  to  provide  practitioners  with  theory-based  and  practical  design  theory  to  achieve  better 
learning  outcomes  (Reigeluth,  2003). 


Purpose  of  the  Study 

The  purpose  of  this  study  is  twofold.  First,  this  study  explores  how  learner-centered  instructional 
approaches  and  Internet  technology  can  be  incorporated  into  ESL  language  courses  (more  specifically, 
English  pragmatic  courses)  and  what  issues  should  be  addressed  to  help  instructors  plan  and  teach  online 
courses.  Pragmatic  competence  refers  to  a speaker’s  ability  to  use  English  form  and  meaning  appropriately 
in  a given  context  (Canale,  1983;  Hymes,  1972).  It  is  related  to  a speaker’s  competence  “to  effectively 
address  issues  of  politeness,  formality,  metaphor,  register,  and  culturally  related  aspects  of  a target 
language”  (Carel,  1999,  p.  11).  Pragmatic  competence  of  ESL  learners  is  considered  as  important  as 
grammatical  competence  for  successful  communication  (Kasper,  1997).  However,  research  on  pragmatics 
mainly  focuses  on  “detailed  descriptions  of  realization  strategies  for  perhaps  eight  speech  acts  in  a variety 
of  situations”  (Cohen  & Olshtain,  1993,  p.  34),  while  the  design  of  instruction  for  pragmatics  has  received 
little  research  attention,  and  few  design  theories  have  been  created  to  develop  learners’  pragmatic 
competence.  Although  knowing  pragmatic  theory  and  research  is  important  for  instructors  to  decide  what  to 
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teach  (Kasper,  1997),  the  descriptive  nature  of  pragmatic  theory  makes  it  difficult  for  practitioners  to  apply 
it  to  their  instructional  problems  and  identify  methods  for  a specific  situation  (Reigeluth,  1999). 
Prescriptive  instructional  design  theory,  which  focuses  on  methods  of  instruction,  is  necessary  to  offer 
pedagogical  guidelines  for  the  design  of  online  ESL  pragmatic  courses. 

For  the  second  purpose,  based  on  interview  data,  analysis  of  pragmatic  lessons,  the  researcher’s 
experiences,  and  a broad  review  of  literature,  this  study  is  intended  to  create  an  instructional  design  theory 
which  provides  faculty  in  higher  education  with  practical  guidelines  to  incorporate  user-centered  and 
collaborative  learning  approaches  into  their  entirely  Web-based  courses.  These  pedagogically  sound 
strategies  support  the  target  audiences  to  design  Web-based  instruction  to  increase  ESL  learners’  pragmatic 
awareness  and  assist  them  in  engaging  students  in  their  learning  process  and  improving  their  learning 
outcomes.  The  function  of  new  instructional  and  communication  technology  is  empowered  by  this  design 
theory. 


Learner  Centered  Perspectives 

For  this  study,  the  learner-centered  perspective  is  selected  as  a design  approach  to  develop  a new 
instructional  design  theory.  As  a researcher  who  tries  to  find  better  ways  to  facilitate  learning,  the 
researcher  would  rather  take  pragmatic  and  eclectic  approaches.  That  is,  the  goal  of  this  research  is  to  draw 
useful  methods  from  diverse  theoretical  perspectives  to  achieve  stated  learning  goals  (Reigeluth,  1992; 
1999).  For  this  study,  the  learner-centered  perspective  does  not  refer  to  one  specific  educational  perspective 
or  philosophy  or  set  of  principles.  Rather,  it  refers  to  a movement  started  by  many  researchers  who  raised 
questions  about  the  instructional  effectiveness  of  traditional  teacher-led  approaches  (APA,  1997;  Bonk  & 
Cunningham,  1998;  Bransford  et  al.,  1990;  Cuban,  1993;  Duffy  & Jonassen,  1992;  Freire,  1972;  McCombs, 
2001;Reigeluth,  1999;  Perkins,  1992;  Wagner  & McCombs,  1995).  From  their  perspectives,  it  is  viewed 
that  the  learner-centered  approaches  place  learners  and  their  learning  in  the  center  of  the  instruction,  value 
learners’  needs,  and  emphasize  learning  and  their  active  roles  in  their  learning  processes. 

Reigeluth  (1999)  discusses  learning-focused  instruction  as  a new  paradigm  of  education  in 
contrast  to  sorting-focused  instruction  as  an  old  paradigm  of  education.  He  pays  attention  to  changing 
needs  of  the  new  society  by  delineating  the  key  markers  of  the  industrial  age  and  the  information  age. 
According  to  Reigeluth,  the  current  system  of  education  is  based  on  standardization,  conformity,  and  the 
parts-oriented  paradigm  of  the  industrial  age,  and  it  fails  to  prepare  students  for  new  abilities  that  the 
information  age  requires,  such  as  problem  solving  skills,  collaborative  learning  skills,  and  self-regulated 
skills. 

The  two  approaches  (traditional  and  learner-centered)  are  incompatible  perspectives  in  terms  of 
learning  and  the  way  each  perspective  fosters  learning.  The  traditional  approach,  grounded  in  the 
objectivist  philosophy,  assumes  that  knowledge  can  be  transferred  from  teachers  to  students  (Duffy  & 
Jonassen,  1992).  In  class,  instructors  focus  on  the  delivery  of  predetermined  knowledge  and  help  students  to 
master  the  knowledge.  The  types  of  learning  that  instructors  teach  are  mainly  knowledge  and  understanding 
levels.  During  the  learning  process,  instructors  have  the  authority  to  decide  and  control  course  content, 
objectives,  strategies,  and  assessment  for  students  (Wagner  & McCombs,  1995).  At  that  time,  learner 
differences  and  needs  are  often  overlooked  for  logistical  and  economical  purposes  (Reigeluth,  1999). 

In  contrast,  learner-centered  approaches,  which  are  highlighted  by  the  Learner-Centered 
Psychological  Principles  of  American  Psychological  Association  (APA),  show  different  perspectives  from 
traditional  approaches.  The  APA’s  learner-centered  principles  provide  “a  knowledge  base  for 
understanding  learning  and  motivation  as  natural  processes  that  occur  when  the  conditions  and  context  of 
learning  support  individual  learner  needs,  capacities,  experiences,  and  interests.”  (McCombs,  2001,  p.  185). 
The  fourteen  learner-centered  principles  are  categorized  into  four  domains:  metacognitive  and  cognitive 
factors;  affective  and  motivational  factors;  developmental  and  social  factors;  and  individual  difference 
factors.  The  metacognitive  and  cognitive  factors  draw  attention  to  the  importance  of  knowledge 
construction  through  nurturing  meaningful  experiences,  goal-directed  and  self-regulated  learning,  strategic 
learning  skills,  higher-order  metacognitive  strategies,  and  supportive  learning  contexts.  To  accomplish 
these  learning  goals,  the  principles  focus  on  instruction  which  promotes  student  motivation,  reflects 
individual  differences  in  learning,  and  nurtures  social  interactions. 

Learner-centered  approaches  can  be  summed  up  as  follows:  in  a learner-centered  classroom, 
learners  are  no  longer  passive  recipients  of  decontextualized  skills  and  knowledge  transmitted  by 
instructors.  They  are  encouraged  to  be  active  participants,  co-producers,  and  disseminators  of  knowledge  in 
their  learning  process.  Instruction  helps  learners  learn  through  interactions  with  peers,  instructors,  and 
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experts.  They,  as  a group,  engage  in  tasks  which  are  more  personally  relevant  to  them.  At  this  time, 
instruction  promotes  learners’  curiosity  and  intrinsic  motivation  to  help  them  link  new  information  to  what 
they  already  know.  As  a result,  shared  knowledge  is  generated  and  disseminated  among  members. 
Traditionally  an  instructor  is  the  only  one  who  has  authority  over  the  instructional  process.  In  a learner- 
centered  classroom,  learners  are  asked  to  bring  their  own  needs  and  experiences,  and  instructors  address 
their  needs  in  instruction  and  strategies  to  promote  learners’  ownership  and  responsibility  for  learning.  As  a 
result,  the  learner-centered  perspective  creates  more  meaningful  and  dynamic  environments. 

Researchers  advocate  that  learner-centered  instructional  techniques  embedded  in  Web  based 
instruction  have  many  benefits  (Bonk  & Cummings,  1998;  Miller  & Miller,  2000;  Warchauer,  2000).  They 
argue  that  it  generates  authentic  learning  environments  that  help  engage  learners  in  personally  meaningful 
activities  and  provide  instructors  with  a variety  of  methods  to  accommodate  individual  student  learning 
styles  and  needs.  For  instance,  Bonk  and  Cummings  (1998)  used  APA  principles  to  design  and  implement 
Web-based  educational  psychology  courses  offered  to  preservice  teachers  at  Indiana  University.  They 
report  that  learner-centered  pedagogical  activities  used  in  the  course  support  student  participation,  foster 
student  thinking  skills,  problem  solving  abilities,  teamwork,  and  social  interaction  and  debate.  They  also 
state  that  the  approach  helps  students  to  have  opportunities  to  experiment  with  their  instructional  ideas  and 
reflect  on  the  teaching  and  learning  activities  they  observe  from  field  experiences.  From  the  experiences, 
they  suggest  12  recommendations  for  Web-based  instruction.  Their  guidelines  include: 

establishing  a safe  learning  community,  fostering  student  engagement,  giving  students  choice, 
facilitating  learning,  offering  public  and  private  feedback,  apprenticing  student  learning, 
employing,  recursive  assignments,  utilizing  electronic  writing  and  reflection  activities,  building  on 
student  Web  link  suggestions,  providing  clear  expectations  and  prompt  task  structuring,  evaluating 
student  electronic  work,  and  personalizing  learning  as  technology  advances  (p.  82-89). 

As  presented,  learner-centered  perspectives  and  Web  technology  have  the  potential  to  work 
synergistically  to  overcome  the  limitations  of  teacher-led  approaches.  Considering  that  the  goal  of  this 
research  is  to  design  a Web-based  guideline  for  developing  students’  pragmatic  awareness,  methods  that 
encourage  rote  memorization  of  facts  and  rules  cannot  turn  learners  into  successful  language  users  (Kern  & 
Warchauer,  2000;  Reynard,  2003).  Providing  natural  and  meaningful  interactions  in  an  authentic  context 
should  be  the  key  conditions  for  the  course  design. 

Methodological  Process 

This  research  started  with  a pilot  study  in  which  the  researcher  interviewed  language  instructors 
and  faculty  teaching  English,  Spanish,  or  French  to  second-language  learners  in  higher  education  in  the 
United  States.  The  interviews  helped  the  researchers  to  understand  the  current  face-to-face  instructional 
language  learning  situations  (including  ESL  pragmatic  learning)  and  their  interests  and  concerns  about  the 
use  of  Web-based  instruction  for  their  courses.  The  interview  data  showed  that  there  were  some  constraints 
for  the  current  face-to-face  classrooms  to  develop  learners’  ESL  pragmatic  skills.  The  limitations  were 
linked  to  the  current  curriculum,  time,  lack  of  knowledge  of  methods,  inappropriate  ESL  textbooks,  passive 
students,  and  lack  of  interactions  in  terms  of  authenticity,  quality,  and  quantity.  When  asked,  many 
instructors  showed  their  interest  in  WBI  and  willingness  to  participate  in  teaching  online  if  the  chance  were 
available,  whereas  some  of  them  expressed  concerns  about  a few  pedagogical  guidelines,  their  technology 
skills,  and  lack  of  interactions  among  participants  in  Web-based  courses. 

In  response  to  these  needs  and  concerns,  this  research  study  has  developed  this  IDT  from  the 
learner-centered  instructional  perspective  to  assist  instructors  to  integrate  Web  technology  and  resources 
into  their  entirely  Web-based  ESL  courses. 

Creation  of  this  IDT  has  been  guided  by  the  five-stage  theory  development  process  of  the 
formative  research  methodology  (Reigeluth  & Frick,  1999).  The  process  included  the  following  phases: 

(1)  Create  a case  to  help  generate  the  design  theory 

In  this  phase,  the  case  included  the  design  of  the  online  pragmatic  course  (one-credit, 
supplementary  English  course  for  graduate  and  undergraduate  students)  and  the  IDT  side  by  side.  The 
initial  IDT  has  been  developed  inductively  based  on  various  resources:  interview  data  from  ESL  faculty 
and  instructors,  analysis  of  university  ESL  pragmatic  lessons,  the  researcher’s  teaching  experiences,  and  a 
broad  review  of  theory  and  research  findings  from  learning  theory,  instructional  design  theory,  Web-based 
learning,  second-language  acquisition,  and  second-language  pragmatics. 

(2)  Collect  and  analyze  formative  data  on  the  instance 
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For  the  theory  revision,  seven  experts  from  WBI  teaching  and  design  and  second  language  education, 
including  the  TESOL,  Spanish,  and  Italian  language  fields,  were  invited  to  review  the  IDT  and  asked  to 
make  suggestions  for  ways  of  refining  it.  Multiple  data  collection  rounds  were  used  until  the  researcher 
obtained  consistency  of  results  among  participants. 

(3)  Revise  the  instance 

The  initial  prototype  model  was  revised  based  on  the  formative  data  from  the  expert  review. 

(4)  Repeat  the  data  collection  and  revision  cycle 

The  second  and  third  rounds  of  the  data  collection  and  revision  were  conducted. 

(5)  Fully  develop  the  tentative  theory 

Based  on  the  data  from  the  expert  review,  the  initial  theory  has  been  revised  and  finalized. 

Outline  of  the  Initial  IDT 

This  initial  IDT  consists  of  the  key  components  of  instructional  design  theory  suggested  by 
Reigeluth  (1999):  (1)  instructional  conditions,  (2)  instructional  values,  and  (3)  instructional  methods.  The 
instructional  conditions  of  this  IDT  include  the  content  to  be  learned,  learner’s  and  instructor’s 
characteristics,  and  learning  environment  and  constraints.  The  values  of  this  IDT  include:  (1)  learner- 
centeredness,  (2)  customized  learning,  (3)  dynamic  learning  through  collaboration,  and  (4)  supportive  and 
respectful  relationship. 

The  method  sections  of  the  initial  IDT  are  structured  by  four  key  design  methods,  which  provide 
detailed  sub  strategies  and  guide  users  to  decide  when  to  use  and  how  to  use  the  suggested  strategies  for  a 
supportive,  authentic,  collaborative,  and  customized  learning  experience.  The  first  design  method  aims  at 
providing  technological,  pedagogical,  and  social  supports  to  help  students  to  acclimate  to  new  instructional 
environments.  The  second  design  method  suggests  means  to  utilize  Web  resources  as  authentic  materials 
and  to  structure  activities  to  be  situated  in  real  world  contexts.  The  third  design  method  concerns  strategies 
for  fostering  collaborative  learning  through  a sense  of  community  among  participants  and  the  use  of 
synchronous  and  asynchronous  activities.  The  forth  design  methods  suggests  a means  to  nurture  learner 
control  and  motivation. 


Findings  and  Revisions  from  the  Expert  Review 

The  initially  developed  IDT  was  tested  formatively  by  seven  experts  representing  WBI  and 
second-language  education  fields.  Through  multiple  data  collection  rounds,  the  researcher  was  able  to 
obtain  constructive  comments  and  suggestions  about  the  initial  theory.  The  identified  weaknesses  of  the 
methods  have  been  addressed  as  follows  (Table  1). 


Table  1 Findings  from  the  Expert  Review  and  Subsequent  Modifications 


Category 

Issues  on  Methods 


Issues  on 
Organization, 
Clarification, 


> 

> 


> 

> 


> 

> 


Identified  Weaknesses  & Modifications 
Clarify  the  learner-centered  perspective  used  in  this  IDT. 

Suggest  methods  maximizing  customized  learning  while  dealing  with 
instructors’  workload  and  students’  concern  for  grading. 

Promote  authentic  and  elaborate  inputs  and  also  caution  against  challenges  in 
using  authentic  resources. 

Address  the  development  of  the  verbal  pragmatic  competency. 

Recommend  multi-evaluation  strategies  to  handle  issues,  such  as  free-riding, 
inactive  participation,  and  complaints  about  grading. 

Incorporate  mid-term  evaluation  to  identify  students’  perceived  problems  and 
attitudes  towards  instruction  during  the  semester. 

Reemphasize  the  importance  of  ground  rules  in  order  to  increase  mutual 
understandings  between  students  and  instructors  about  the  course. 

Highlight  methods  promoting  meaningful  interactions  and  collaborations  among 
participants. 

Reemphasize  the  importance  of  learning  community  and  the  benefits  of 
synchronous  communication  to  foster  a strong  sense  of  community. 

Suggest  new  tools  for  fostering  synchronous  communications. 

Increase  connections  between  the  suggested  methods  and  ESL  pragmatic 
learning  contexts,  and  focus  the  connection  among  the  three  key  design  methods 
of  this  IDT. 
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&Redundancy  > Provide  additional  information  related  to  the  background  of  this  IDT  and  field- 
specific  terminologies. 

> Clarify  ambiguous  wording  and  structure. 

> Remove  redundant  and  overlapping  components  and  methods. 

Outline  of  the  Revised  IDT 

The  framework  of  the  IDT  remains  unchanged.  It  consists  of  instructional  conditions,  instructional 
values,  and  instructional  methods.  In  contrast,  some  parts  of  the  methods  section  have  been  changed 
significantly  to  increase  links  between  the  key  design  methods  and  the  focused  learning  contexts  and  to 
elaborate  the  initial  methods.  The  first  key  design  method  concerns  learning  strategies  that  help  students  to 
acclimate  to  new  online  instructional  environments  that  are  different  from  typical  face-to-face  teacher-led 
classrooms.  The  second  key  design  method  presents  sub- strategies  for  enhancing  ESL  pragmatic  learning, 
including:  (1)  choose  appropriate  topics  and  tasks  which  are  closely  linked  to  ESL  learners,  (2)  use 
authentic  resources,  (3)  promote  interaction  and  collaboration,  (4)  provide  constructive  and  timely 
scaffolding  and  feedback,  and  (5)  use  multiple  assessment  strategies.  The  third  key  design  method 
recommends  a means  to  nurture  learner  control  and  motivation  promoting  independent  and  motivational 
learning  in  Web-based  environments. 

Future  users  of  this  IDT  are  strongly  encouraged  to  adapt  it  to  accommodate  their  unique 
instructional  situations  considering  special  needs  of  students  and  their  current  levels  of  understanding  and 
performance.  Users’  active  modification  will  contribute  to  make  this  IDT  more  usable  and  practical. 
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Abstract 

A traditional  Task  Analysis  (TA)  usually  involves  conducting  data  collection  through  focus 
groups , observations,  interviews  and  review  of  existing  courseware  and  documentation.  At  a minimum,  the 
TA  focuses  on  the  capturing  of  specific  data  elements,  knowledge,  skills,  and  attitudes  (KSAs),  along  with 
difficulty,  importance,  and  frequency  (DIF)  indicators  and  conditions  and  standards.  These  elements  may 
be  stored  as  either  paper-based  products  or  in  electronic  means,  as  documents,  spreadsheets  or  local 
database  driven  applications.  However,  such  a traditional  approach  to  conducting  a TA  has  its  challenges. 

The  level  of  difficulty  involved  is  dependent  upon  the  scope  of  the  TA  in  question.  There  may  be 
coordination  and  management  issues  due  to  the  size  of  the  effort  and  the  number  of  individuals  (subject 
matter  experts  (SMEs),  facilitators ) and  support  personnel ) involved.  Administrative  support  issues  may 
also  come  into  play,  such  as  the  logistics  of  meeting  locations,  breakout  rooms,  flight  schedules  and 
preparation  of  printed  and  electronic  presentation  materials.  The  issues  may  become  even  greater  if  the 
participants  involved  are  distributed  remotely,  leading  to  scheduling  complications  for  face-to-face 
sessions.  Staffing  and  administrative  factors  aside,  the  breadth  of  the  data  to  be  collected  may  turn  into  a 
configuration  management  nightmare  on  its  own.  All  of  these  factors  constrain  the  collaboration  process. 

So,  how  can  collaborative  efforts  be  successfully  managed  within  a TA  environment?  First  and 
foremost,  advancements  in  technology  provide  the  backbone  for  large-scale  TA  projects  to  incorporate 
collaborative  tools,  such  as  synchronous,  asynchronous,  automated/pre-programmed  and  one-way 
communication  styles.  Programs,  such  as  the  Army's  Future  Combat  Systems  (FCS)  and  the  Navy's 
Transformation  in  Training,  have  already  begun  the  successful  use  of  custom  web-enabled  tools  to 
automate  the  TA  and  collaboration  processes  from  the  ground  up.  For  example,  beginning  with  the  end 
result  in  mind  and  the  participants  involved,  web -enabled  tools  have  been  developed  for  FCS  to  aid  in  the 
compilation,  management,  storage,  reporting  and  most  importantly,  collaboration  in  the  collection  of  data 
and  metadata  for  a TA.  In  such  a case,  participants,  such  as  program  managers,  team  leads  and  SMEs  are 
collaborating  successfully  and  effectively  from  remote  locations  across  the  nation.  Large-scale  projects 
involved  in  these  analysis  efforts  are  seeing  the  benefits  of  seamlessly  incorporating  both  over-the-counter 
software  packages,  known  as  shrink-wrap,  and  custom-developed  software  solutions. 

Secondly,  the  collaboration  effort  itself  must  be  examined.  The  methods  of  collaboration  may  be 
different  and  dependent  upon  the  effort  at  hand  and  the  needs  of  the  program.  Determining  the  right 
methods  of  collaboration  will  result  from  asking  the  right  questions.  How  will  those  involved  in  the  analysis 
effort  collect  information?  How  must  the  information  be  stored  and  shared?  What  exactly  will  be  required 
for  interaction— face-to-face  or  through  other  remote  means?  Namely,  to  what  degree  will  there  be 
collaboration  and  on  what  levels? 

This  paper  focuses  on  three  key  issues  promoting  automated  collaboration  within  a large-scale  TA 
effort.  Large-scale  equates  to  multiple  distributed  sites,  various  teams,  the  use  of  metrics  (e.g.,  keeping 
track  of  task  progress  or  establishing  automated  reviews)  and  the  collection  ofTA  data  and  metadata. 

These  key  issues  are: 

• The  challenges  in  automating  a successful  collaboration  process.  In  particular,  the  need  for 
automated  tools  (shrink-wrap  and  custom)  to  facilitate  a seamless  process  is  examined. 

• The  communication  structure  itself.  Namely,  the  most  common  styles  of  communication  needed  in 
such  automation. 

• The  implementation  of  these  communication  styles  as  software. 

Additionally,  usability  (ease  of  use  for  the  end  user)  will  ensure  the  success  of  this  process  in  such 
complex  automated  tools.  Essentially,  what  steps  can  be  taken  early  in  the  development  of  such  tools  to 
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ensure  they  are  indeed  applicable  to  the  effort  at  hand.  Should  human  factors  play  a key  role  in  providing  a 
liaison  between  that  of  development  and  the  end-user?  These  issues  will  be  examined. 


Introduction 

Lastly,  the  TA  data  itself  (which  has  been  captured  using  groupware  and  database-driven 
technologies)  is  the  basis  of  an  Instructional  Designer’s  design  and  development  strategy.  So,  why  not 
leverage  this  same  technology  so  it  aids  the  Instructional  Designer  in  providing  Instructional  System 
Design  (ISD)  capabilities,  such  as  storyboarding  and  authoring?  Why  should  the  designer  have  to  pull  this 
data  from  one  system  into  another  (or  multiple  systems),  in  the  process  inheriting  migration  costs,  so  that 
the  data  can  be  analyzed  and  developed?  Depending  on  the  TA  data  and  groupware  capabilities, 
Instructional  Designers  may  be  able  to  define  what  to  train,  what  type  of  medium  to  train  within  and  to 
guide  designers  in  creating  a course  outline.  And,  if  enough  information  is  contained  with  the  TA  data,  it 

A traditional  Task  Analysis  (TA)  usually  involves  conducting  data  collection  through  focus 
groups,  observations,  interviews  and  review  of  existing  courseware  and  documentation.  At  a minimum,  the 
TA  focuses  on  the  capturing  of  specific  data  elements,  knowledge,  skills,  and  attitudes  (KSAs),  along  with 
difficulty,  importance,  and  frequency  (DIF)  indicators  and  conditions  and  standards.  These  elements  may 
be  stored  as  either  paper-based  products  or  in  electronic  means,  as  documents,  spreadsheets  or  local 
database  driven  applications.  However,  such  a traditional  approach  to  conducting  a TA  has  its  challenges. 

The  level  of  difficulty  involved  is  dependent  upon  the  scope  of  the  TA  in  question.  There  may  be 
coordination  and  management  issues  due  to  the  size  of  the  effort  and  the  number  of  individuals  (subject 
matter  experts  (SMEs),  facilitator(s)  and  support  personnel)  involved.  Administrative  support  issues  may 
also  come  into  play,  such  as  the  logistics  of  meeting  locations,  breakout  rooms,  flight  schedules  and 
preparation  of  printed  and  electronic  presentation  materials.  The  issues  may  become  even  greater  if  the 
participants  involved  are  distributed  remotely,  leading  to  scheduling  complications  for  face-to-face 
sessions.  Staffing  and  administrative  factors  aside,  the  breadth  of  the  data  to  be  collected  may  turn  into  a 
configuration  management  nightmare  on  its  own.  All  of  these  factors  constrain  the  collaboration  process. 

So,  how  can  collaborative  efforts  be  successfully  managed  within  a TA  environment?  First  and 
foremost,  advancements  in  technology  provide  the  backbone  for  large-scale  TA  projects  to  incorporate 
collaborative  tools,  such  as  synchronous,  asynchronous,  automated/pre-programmed  and  one-way 
communication  styles.  Programs,  such  as  the  Army’s  Future  Combat  Systems  (FCS)  and  the  Navy’s 
Transformation  in  Training,  have  already  begun  the  successful  use  of  custom  web-enabled  tools  to 
automate  the  TA  and  collaboration  processes  from  the  ground  up.  For  example,  beginning  with  the  end 
result  in  mind  and  the  participants  involved,  web-enabled  tools  have  been  developed  for  FCS  to  aid  in  the 
compilation,  management,  storage,  reporting,  and  most  importantly,  collaboration  in  the  collection  of  data 
and  metadata  for  a TA.  In  such  a case,  participants,  such  as  program  managers,  team  leads  and  SMEs  are 
collaborating  successfully  and  effectively  from  remote  locations  across  the  nation.  Large-scale  projects 
involved  in  these  analysis  efforts  are  seeing  the  benefits  of  seamlessly  incorporating  both  over-the-counter 
software  packages,  known  as  shrink-wrap,  and  custom-developed  software  solutions. 

Secondly,  the  collaboration  effort  itself  must  be  examined.  The  methods  of  collaboration  may  be 
different  and  dependent  upon  the  effort  at  hand  and  the  needs  of  the  program.  Determining  the  right 
methods  of  collaboration  will  result  from  asking  the  right  questions.  How  will  those  involved  in  the 
analysis  effort  collect  information?  How  must  the  information  be  stored  and  shared?  What  exactly  will  be 
required  for  interaction — face-to-face  or  through  other  remote  means?  Namely,  to  what  degree  will  there  be 
collaboration  and  on  what  levels? 

This  paper  focuses  on  three  key  issues  promoting  automated  collaboration  within  a large-scale  TA 
effort.  Large-scale  equates  to  multiple  distributed  sites,  various  teams,  the  use  of  metrics  (e.g.,  keeping 
track  of  task  progress  or  establishing  automated  reviews)  and  the  collection  of  TA  data  and  metadata.  These 
key  issues  are: 

• The  challenges  in  automating  a successful  collaboration  process.  In  particular,  the  need  for 
automated  tools  (shrink-wrap  and  custom)  to  facilitate  a seamless  process  is  examined. 

• The  communication  structure  itself.  Namely,  the  most  common  styles  of  communication 
needed  in  such  automation. 

• The  implementation  of  these  communication  styles  as  software. 

Additionally,  usability  (ease  of  use  for  the  end  user)  will  ensure  the  success  of  this  process  in  such 
complex  automated  tools.  Essentially,  what  steps  can  be  taken  early  in  the  development  of  such  tools  to 
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ensure  they  are  indeed  applicable  to  the  effort  at  hand.  Should  human  factors  play  a key  role  in  providing  a 
liaison  between  that  of  development  and  the  end-user?  These  issues  will  be  examined. 

Lastly,  the  TA  data  itself  (which  has  been  captured  using  groupware  and  database-driven 
technologies)  is  the  basis  of  an  Instructional  Designer’s  design  and  development  strategy.  So,  why  not 
leverage  this  same  technology  so  it  aids  the  Instructional  Designer  in  providing  Instructional  System 
Design  (ISD)  capabilities,  such  as  storyboarding  and  authoring?  Why  should  the  designer  have  to  pull  this 
data  from  one  system  into  another  (or  multiple  systems),  in  the  process  inheriting  migration  costs,  so  that 
the  data  can  be  analyzed  and  developed?  Depending  on  the  TA  data  and  groupware  capabilities, 
Instructional  Designers  may  be  able  to  define  what  to  train,  what  type  of  medium  to  train  within  and  to 
guide  designers  in  creating  a course  outline.  And,  if  enough  information  is  contained  with  the  TA  data,  it 
can  serve  as  appropriate  data  for  creating  learning  objectives  and  content. 

The  Challenges 

Advancements  in  technology  provide  the  backbone  for  large-scale  TA  efforts  to  promote 
communication.  Through  the  integration  of  collaborative  tools,  otherwise  known  as  groupware, 
communication  can  be  elevated  onto  a new  level,  incorporating  vehicles  to  promote  successful 
collaboration  between  participants  that  was  once  unachievable  via  traditional  means.  In  some  instances, 
shrink-wrap  groupware  may  be  available,  providing  an  out-of-the-box  solution  that  can  be  cost-effective 
and  quickly  integrated.  In  other  instances,  custom  tools  may  need  to  be  developed,  providing  custom  made 
solutions  to  the  challenges  at  hand.  Regardless  to  whether  shrink-wrap  or  custom  groupware  is  used,  both 
have  their  share  of  challenges. 

Let’s  begin  with  shrink-wrap  groupware.  There  are  a number  of  over-the-counter  software 
packages  available  providing  capabilities  such  as  email,  instant  messaging  and  video/audio  conferencing. 
This  groupware  in  some  instances  may  be  inexpensive  to  purchase,  or  depending  on  the  licensing,  may  be 
available  for  free  commercial  use.  In  other  cases,  groupware,  providing  the  same  or  similar  capabilities, 
may  already  be  available  as  part  of  the  TA  infrastructure,  such  as  the  ability  to  send  and  receive  email. 
Given  the  nature  of  the  groupware  and  popularity,  participants  may  already  be  familiar  with  the  technology, 
requiring  little  or  no  training.  It  may  also  meet  less  resistance  in  its  use  by  the  participants  because  of  their 
comfort  level  with  the  software  as  opposed  to  the  sometimes  common  resistance  met  by  participants  who 
are  unfamiliar  with  a technology  and  as  a result  simply  refuse  to  learn  how  to  use  it. 

The  downside  to  using  shrink-wrap  groupware  is  the  simple  fact  that  it  may  not  completely  meet 
the  collaborative  needs  of  the  TA  effort.  This  may  require  the  use  of  more  than  one  software  product  or  a 
suite  of  products.  The  other  complication  is  ensuring  the  seamless  integration  of  the  groupware  in  such  a 
way  that  it  aids  in  collaboration  without  hindering  it.  The  groupware  should  help  facilitate  communication 
within  the  effort.  It  should  not  impose  constraints,  additional  requirements,  or  force  participants  to  perform 
steps  that  would  not  normally  have  to  be  performed  when  communicating. 

Custom  groupware,  on  the  other  hand,  if  developed  properly  with  the  participants  and  needs  of  the 
effort  in  mind,  should  integrate  seamlessly.  It  should  provide  customized  solutions  to  meet  the  exact  needs 
of  the  TA  effort  allowing  the  participants  to  communicate  with  ease.  Depending  on  the  collaboration 
required,  the  custom  groupware  might  consist  of  one  product  or  multiple  products  using  a common 
framework.  For  example,  the  groupware  may  be  a content  management  system  allowing  multiple 
participants  at  remote  locations  to  collaborate  on  the  development  of  tasks  via  tools  incorporated  into  the 
groupware  system.  Once  authenticated,  participants  can  simply  use  the  system  to  communicate  with  each 
other  while  they  work,  without  the  need  to  use  external  means. 

Custom  groupware  does  have  its  downside,  however.  First  of  all,  this  groupware  must  be 
developed,  in  most  cases  resulting  in  a much  higher  price  tag  than  that  of  shrink-wrap.  The  development  of 
this  groupware  can  become  an  effort  in  itself,  requiring  an  analysis  of  the  TA  effort  clearly  outlining  the 
types  of  collaboration  needed,  a thought  out  design,  time  to  develop  and  document  (online  help),  integrate, 
and  finally,  allow  the  participants  involved  to  evaluate.  This  doesn’t  even  include  the  cost  and  time  to  train 
the  participants  on  its  use.  Although  training  of  participants  is  not  the  main  focus  of  this  paper,  it  is  highly 
recommended  to  set  aside  the  time  to  train  “end-users”  and  to  have  available  either  online  help  or  a job  aid, 
or  both,  on  proper  use.  In  addition,  reach-back  to  the  development  team,  for  participants  who  are  having 
difficulties  in  using  the  tool,  should  be  a consideration  as  well.  All  of  these  steps  must  occur  well  in 
advance  of  the  groupware’s  anticipated  use. 

The  development  team  may  also  have  to  be  used  during  the  TA  effort  as  “bugs”  are  found  in  the 
groupware.  If  new  requirements  are  introduced  during  the  effort,  the  groupware  must  be  modified  and/or 
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new  functionality  added.  It  may  also  cause  potential  delays  in  the  TA  effort  as  the  modifications/new 
features  are  integrated  into  the  groupware.  Not  to  mention  the  time  to  train  the  participants  on  the  new 
capabilities  and  changes.  There  may  also  be  opposition  to  its  use,  particularly  if  the  groupware  is  not 
intuitive.  Participants  may  simply  gravitate  to  using  other  software  and  methods  they  are  comfortable  with 
or  simply  take  the  stance  “. . .we’re  doing  it  this  way.  This  is  the  way  we’ve  always  done  it.” 

One  of  the  most  important  lessons  learned  in  the  integration  of  collaborative  groupware,  whether 
it’s  purchased  off-the-shelf  or  custom  developed,  is  that  recruitment  must  occur  early  and  across  the 
different  echelons  of  participants  involved,  from  management  to  end-users.  In  order  to  be  more  successful 
in  getting  positive  participation,  the  purpose  of  the  use  of  groupware  must  be  clearly  established,  outlining 
the  exact  benefits  of  using  such  groupware.  In  other  words,  what  is  the  return  on  investment?  Is  it 
monetary?  Is  the  result  more  efficiency  or  less  time  to  perform  a task?  How  will  the  groupware  help  the 
participants?  Why  should  the  participants  be  on  board  with  using  this  technology? 

The  list  of  challenges  could  go  on  and  on.  In  fact,  the  challenges  in  automating  a successful 
collaboration  process  through  the  use  of  groupware  merits  its  own  paper.  Knowing  the  challenges  and  the 
lessons  learned  is  the  first  step  in  understanding  how  to  overcome  them.  So,  how  exactly  does  one  do  this? 
The  answer  depends  entirely  on  the  needs  of  the  TA  effort.  Later  in  this  paper,  we’ll  point  out  some  of  the 
most  common  lessons  learned.  First,  however,  we’ll  be  examining  the  communication  structure  itself — 
namely,  the  most  common  styles  of  communication  needed  in  such  automation. 

Styles  of  Communication 

Introducing  technology  into  a TA  effort  without  understanding  how  the  participants  will 
communicate  will  hamper  productivity  and  frustrate  those  involved.  Technology  is  but  one  part  of  the 
equation.  Collaboration  is  only  achieved  when  the  groupware  is  successfully  married  with  the  processes  in 
place,  and  information  can  be  shared  as  closely  as  possible  in  a real-time  fashion.  This  must  occur  across 
all  the  locations  and  with  all  participants  involved.  So,  understanding  the  different  styles  of  communication 
employed  is  also  needed.  Books  have  been  written  on  the  subject  of  communication  ranging  from  its 
history  to  cultural  studies,  and  so  a deep  understanding  of  such  is  out  of  the  scope  of  this  paper.  What  is 
discussed,  however,  are  the  most  common  styles  employed  when  automating  collaboration  as  part  of  a 
large-scale  TA  effort.  Let’s  begin  by  examining  synchronous  and  asynchronous  communication  styles. 

Synchronous  Style 

This  is  a style  of  two-way  communication  between  participants  that  appears  to  occur  with  no  time 
delay.  It  allows  the  participants  to  communicate  (respond  to  each  other)  in  real-time.  By  real-time,  we  mean 
there  is  no  noticeable  time  delay,  even  though  in  actuality  one  may  exist.  To  the  participants,  they  are 
communicating  as  if  they  where  simply  speaking  to  each  other  face-to-face. 

Asynchronous  Style 

The  asynchronous  style,  on  the  other  hand,  is  a two-way  communication  that  occurs  with  a 
noticeable  time  delay.  Participants  know  they  are  not  communicating  in  real-time.  This  style  is  also  less 
stringent  than  synchronous.  Asynchronous  communication  allows  participants  to  respond  at  their  own 
convenience. 

From  an  automation  standpoint,  communication  can  be  achieved  using  these  two  styles.  Most 
groupware  used  today  still  appears  to  be  asynchronous,  even  though  this  seems  to  be  changing  with 
enhancements  such  as  instant  messaging  and  voice  over  IP  technologies.  The  synchronous  and 
asynchronous  styles  provide  two-way  communication,  but,  are  not  enough  when  automating  collaboration 
as  part  of  a TA  effort.  One-way  communication  is  also  needed;  namely,  automated/pre-programmed  and 
one-way  communication  styles. 

Automated/Pre-Programmed  Style 

Similar  to  the  asynchronous  communication  style,  automated/pre-programmed  appears  to  occur  to 
the  participants  with  a noticeable  time  delay.  Unlike  both  the  synchronous  and  asynchronous 
communication  styles,  automated/pre-programmed  provides  no  means  for  the  participant  to  respond.  It  is 
strictly  a one-way  communication  style.  In  addition,  it  is  not  participant  controlled.  Meaning,  participants 
cannot  initiate  the  communication  or  control  when  the  communication  occurs.  (This  is  important,  simply 
because  it  is  the  only  difference  between  automated/pre-programmed  and  the  one-way  communication 
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styles.)  This  style  is  mostly  used  in  management  or  system  level  activities,  such  as  in  the  emailing  of 
notifications  (e.g.  notifications  that  a new  release  of  the  groupware  software  is  now  available). 


One-Way  Style 

Very  similar  to  the  automated/pre-programmed  communication  style,  one-way  communication 
occurs  with  a noticeable  time  delay.  It  provides  no  means  for  participants  to  respond.  However,  it  is 
participant  controlled.  Participants  have  full  control  as  to  when  the  communication  takes  place.  One  such 
example  would  be  a report  of  milestone  status  to  project  managers. 

Implementing  the  Communication  Styles 

Understanding  your  TA  effort  and  the  communication  styles,  which  will  be  used,  is  vital  to 
implementing  the  right  technologies.  As  we’ve  discussed,  though,  this  is  not  necessarily  a simple  task.  So 
far,  we  have  addressed  some  of  the  challenges  in  automating  a successful  collaboration  process  along  with 
the  styles  of  communication  found  in  most  large-scale  TA  efforts.  In  this  section,  we  identify  these 
communication  styles  and  applicable  technology.  Table  1 displays  a matrix  organizing  technology  by 
communication  style. 


Table  1:  Technology  by  Communication  Styles 


Communication  Style 

Technology 

Synchronous  Style 

• Video/Audio/Data  Conferencing 

• Electronic  Whiteboards 

• Application  and  File  Sharing 

• Guided  Web  Browsing 

• Streamed  Media 

• Telephony 

• Instant  Messaging 

• Chat  Rooms 

Asynchronous  Style 

• Discussion  Forums/Boards 

• Email 

• Mailing  Lists 

• Newsgroups 

• Stand-Alone  Applications 

• Downloadable  Media 

• Electric  Calendar  Systems 

Automated/Pre-Programmed  Style 

• Automated  Email  Notifications 

One-Way  Style 

• Reporting 

This  is  in  no  way  an  exhaustive  list,  but  lists  those  “types”  of  technologies  with  which  most  are 
familiar.  Keep  in  mind  there  are  a number  of  open  source  and  proprietary  shrink-wrap  products  available 
on  the  market  today.  These  products  can  support  each  of  these  technologies  if  not  spanning  a multitude  of 
them  such  as  in  the  case  of  Lotus  software  by  IBM  ® (IBM,  2005). 

In  providing  an  example  of  how  these  communication  styles  might  be  used  in  conducting  a TA, 
we  will  engage  in  a scenario.  After  the  development  team  meets  to  establish  the  project’s  needs  and  focus, 
channels  are  set  by  which  the  team  members  will  conduct  their  varied  forms  of  communication.  For  initial 
data  collection,  the  Instructional  Designer  and  SME  both  participate  in  the  gathering  of  existing  materials, 
and  must  maintain  in  contact  with  their  progress,  sharing  notes,  questions,  suggestions,  and  decision- 
making on  a continual  basis.  To  this  end,  those  involved  in  the  TA  will  use  all  styles  of  communication. 

In  the  project  scenario,  the  SME  and  Instructional  Designer  may  communicate  synchronously  via 
audio  conferencing  to  coordinate  and  delegate  data  collection  and  file  sharing;  implement  guided  web 
browsing  for  research  purposes;  and  engage  in  instant  messaging  to  enable  immediate  inquiries  and 
feedback.  Asynchronously,  the  SME  and  Instructional  Designer  can  track  their  internal  milestones  in  TA 
via  electronic  calendar  systems;  maintain  continuous  contact  via  email;  and  continue  to  explore  their 
research  via  discussion  boards,  mailing  lists  and  newsgroups.  With  a project  management  application 
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tracking  milestone  achievement,  email  notifications  are  triggered  to  ensure  the  team  members  are  aware  of 
the  next  steps  to  perform  for  the  project,  or  to  assure  all  elements  were  covered  in  the  completed  milestone. 
Upon  milestone  completion,  Task  Analysis  Reports  and  Learning  Analysis  Reports  are  standard  formats  for 
reporting  progress  and  preparation  towards  the  development  of  training. 

Keep  in  mind  that  if  shrink-wrap  is  used,  the  actions  discussed  in  the  previous  scenario  might 
entail  the  SME  using  multiple  software  applications,  some  of  which  may  not  work  well  with  each  other.  It 
may  also  mean  the  participants  may  have  to  be  trained  on  each  application.  If  custom  groupware  is 
developed,  all  of  the  actions  described  may  be  integrated  into  a single  enterprise  solution  providing  all  the 
capacities  required.  This  approach  may  produce  the  best  seamless  results,  but  at  the  same  time  may  be  the 
most  time  consuming  and  costly.  As  a result,  a combination  of  both  shrink-wrap  and  custom  solutions  may 
be  the  middle  ground  leveraging  the  benefits  of  each,  and  at  the  same  time  attempting  to  minimize  costs. 

Usability  and  Evaluation 

Groupware  usability  appears  to  be  a hot  topic.  There  have  already  been  a number  of  publications 
on  the  subject,  which  have  ranged  from  the  usability  of  specific  groupware  systems  to  “How  exactly  do  you 
go  about  evaluating  groupware?”  Of  these  topics,  however,  the  evaluation  of  groupware,  or  lack  there  of 
seems  to  be  getting  the  most  interest.  In  a review  of  groupware  evaluations  (forty-five  papers  total),  Pinelle 
and  Gutwin  (2000)  concluded  that  formal  evaluation  was  lacking  in  one-third  of  the  groupware  system 
evaluations  they  examined,  resulting  in  a further  conclusion  that  there  was  additional  room  for  low-cost  and 
simple  evaluation  techniques. 

So,  why  is  the  evaluation  of  groupware  so  difficult?  Well,  there  are  different  answers  to  this.  Some 
believe  that  it  is  a factor  of  not  knowing  what  evaluation  approach  to  use.  Others  believe  that  many  of  the 
groupware  systems  available  are  too  complex  and  difficult  to  be  properly  evaluated.  Others  insist  that  the 
evaluation  approaches  available  are  not  sufficient.  Generally,  formal  groupware  evaluation  can  be 
approached  in  different  ways.  The  most  common  we  have  experienced  has  been  examining  how 
participants  collaborate  in  the  real  world  as  it  applies  to  the  TA  effort  in  question.  Other  methods  take  more 
scientific  approaches,  using  methods  (e.g.,  automatically,  empirically,  formally  and  informally  (Nielsen, 
1994;  Usability  Inspections,  2001))  to  measure  usability  without  having  to  incur  the  costs  of  shadowing 
multiple  participants  as  they  perform  their  responsibilities  within  the  effort.  In  other  words,  they  evaluate 
groupware  through  usability  inspection  methods  (Steves,  Morse,  Gutwin  & Greenberg,  2001).  (Usability 
inspection  has  been  around  for  a number  of  years.  Countless  papers  and  books  have  been  published  and 
courses  are  available  on  the  subject  and  so  a full  discussion  on  this  topic  is  simply  out  of  the  scope  of  this 
paper.  To  put  it  simply,  usability  inspection  is  nothing  more  than  a generic  way  of  describing  a number  of 
methods  for  finding  usability  problems  with  software  in  the  hopes  of  debugging,  improving  code  and 
enhancing  user  interfaces  (Nielsen,  1994;  Usability  Inspections,  2001).) 

From  our  own  findings  and  experiences,  groupware  evaluation  is  a difficult  problem  to  solve. 
Overall,  there  seems  to  be  little  known  about  the  usability  of  groupware  systems.  What  we  have  learned  in 
the  case  of  custom-developed  groupware  is  that  usability  needs  to  be  examined  early  in  the  development 
process.  We  have  seen  that  those  TA  efforts,  which  ultimately  provided  the  best  use  of  groupware  to 
participants,  were  those  in  which  the  participants  were  involved  during  the  development  of  the  groupware. 
Meaning,  participants  were  involved  in  the  testing  and  evaluation  as  capabilities  and  features  where  ready. 
This  provided  the  opportunity  for  participants  to  use  the  groupware  and  give  feedback  as  to  what  they 
liked,  disliked  and  what  needed  improvement  and  how.  This  approach  not  only  provides  developers 
immediate  feedback  on  development  efforts,  but  also  allows  the  participants  to  become  familiar  with  the 
groupware  and  comfortable  with  the  technology.  This  approach  recruits  participants  very  early  in  the  effort. 

Human  factors  have  also  been  shown  to  be  invaluable  to  these  types  of  efforts,  particularly  in  the 
area  of  graphical  user  interface  development  and  layout.  This  has  helped  in  the  design  of  the  groupware, 
catching  problems  and  usability  issues  early  in  development  prior  to  participant  testing  and  evaluation. 

Involving  developers,  human  factors  experts,  participants  and  SMEs  early  on  does  have  its  share 
of  challenges.  First,  this  approach  assumes  there  is  adequate  time  to  develop  the  groupware  prior  to  the  start 
of  the  TA  effort.  It  also  assumes  that  participants  are  available  for  testing  and  evaluation  during 
development.  This  may  not  always  be  the  case.  Even  if  participants  are  available,  they  may  not  fully 
understand  the  responsibilities  that  will  be  required  of  them.  This  leads  to  inadequate  evaluation  and 
limited  feedback.  This  approach  can  also  be  very  costly. 

To  summarize,  the  following  outlines  what  steps  can  be  taken  early  in  the  development  of 
groupware  to  ensure  they  are  indeed  applicable  to  the  effort  at  hand. 
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• Set  up  meeting(s)  to  identify  stakeholders  and  participants.  Namely,  who  should  support  the 
collection  of  data  and  what  type  of  data  and  elements  should  be  collected.  This  meeting  could  be 
face-to-face  or  conducted  using  synchronous  styles  of  communication  discussed  earlier. 

• Once  these  individuals  are  identified,  designate  a subset  of  these  participants  across  the  various 
functional  areas  as  those  who  will  be  aiding  in  the  testing  and  evaluation  of  the  groupware.  These 
participants  will  act  as  SMEs  and  should  have  enough  knowledge  of  their  functional  area  and  of 
their  responsibilities  within  the  TA  effort  to  speak  for  their  team  who  will  also  be  using  the 
groupware  and  performing  the  same  jobs. 

• Determine  milestones  for  the  collection  of  data.  If  there  are  several  jobs  being  investigated, 
determine  if  the  data  can  be  collected  simultaneously  or  if  one  job  affects  the  data  collected  of  a 
different  job. 

• Develop  groupware  based  on  the  structure  of  the  data  and  the  type  of  access  required  by 
participants  (i.e.,  single  users  or  multiple  users  of  groupware  and  so  forth).  Human  factors  experts 
may  be  of  significant  help  in  this  area  in  the  design  of  user  interfaces  and  other  usability  matters. 

• Use  a spiral  development  approach,  as  milestones  and  major  functionality  are  ready  for  feedback 
by  participants;  conduct  a mock-up  TA  via  the  groupware  in  question.  Work  with  participants  to 
acquire  feedback  on  usability. 

• Evaluate  the  feedback,  prioritizing  requests  that  can  be  easily  integrated  into  the  groupware  system 
first.  Keep  in  mind  that  not  all  the  requests  can  be  implemented  or  done  so  immediately  given  the 
technology,  budget  and  time  constraints. 

As  for  the  usability  of  shrink-wrap  groupware,  our  experiences  have  been  very  similar. 

Participants  should  be  involved  in  the  selection  of  the  groupware  to  be  used  in  the  effort.  This  does  not 
mean  that  they  need  to  be  involved  in  the  initial  trade  study  to  determine  the  potential  groupware  systems 
candidates,  but  should  be  involved  in  the  final  selection  of  one  or  more  of  them  for  use  in  the  TA  effort.  To 
summarize,  the  following  outlines  what  steps  can  be  taken  early  in  the  selection  of  shrink-wrap  groupware 
to  ensure  they  are  indeed  applicable  to  the  effort  at  hand. 

• Set  up  meeting(s)  to  identify  stakeholders  and  participants  (as  described  earlier). 

• Designate  a subset  of  these  participants  to  aid  in  the  selection  of  the  groupware. 

• Determine  milestones  for  the  collection  of  data. 

• Select  the  groupware  systems  based  on  the  structure  of  the  data  and  type  of  access  required  by 
participants.  Keeping  in  mind,  of  course,  that  there  may  be  other  factors  contributing  to  the  section 
as  part  of  the  trade  study  (e.g.,  costs  and  technological  considerations). 

• Allow  participants  to  be  involved  in  the  evaluation  of  the  groupware  systems  determined  to  be  a 
potential  fit  as  part  of  the  trade  study. 

Linkage  of  Task  Analysis  Data  to  the  ADDIE  Model 

The  traditional  instructional  design  process  typically  involves  five  phases:  analysis,  design, 
development,  implementation  and  evaluation,  otherwise  known  as  the  ADDIE  model.  The  TA  data, 
collected  and  stored  using  groupware  and  database  driven  technologies,  is  a big  part  of  the  analysis  phase 
of  the  ISD  process.  How  can  this  data  be  automatically  migrated  to  other  tools  for  the  design  and 
development  phases?  Can  database-driven  groupware  tools  be  linked  to  these  outside  software  packages  in 
order  to  decrease  the  design  and  development  time?  Can  the  same  groupware  be  enhanced  to  perform  the 
capabilities  needed  by  the  Instructional  Designer? 

With  advancement  in  technology,  standardization  being  put  in  place  on  baseline  configuration  of 
computer  systems  (such  as  Distance  Learning  XXI  (Baseline  Home  Computer,  2004)),  and  the  greater  need 
for  software  components  to  work  together,  the  challenge  appears  entirely  achievable.  A basic  TA  includes 
the  elements  of  KSAs,  DIF,  conditions,  standards  and  performance  measures.  So,  why  not  include  elements 
that  allow  the  input  of  learning  objectives  (LOs) — both  terminal  objectives  (TLOs)  and  enabling  objectives 
(ELOs) — to  be  linked  to  the  proper  tasks  or  steps?  In  addition,  the  LOs  can  be  connected  to  the  best 
learning  medium  (a  model  that  is  run  based  on  additional  elements  (bulleted  below)  tied  to  the  LOs). 
Consider  the  following  additional  elements  that  could  be  included  in  the  groupware  for  additional 
automated  processes: 

• Task  Selection  Model,  an  algorithm  using  the  task  difficulty,  importance  and  frequency  data  to 
determine  what  task  to  train  or  not  to  train.  This  helps  to  prioritize  task  development.  For  example, 
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a task  that  is  not  difficult,  not  important  and  infrequently  performed  may  not  require  formal 
training  but  could  be  easily  learned  on-the-job  (Task  Selection  Model,  2005). 

• Learning  objective  fields  could  be  built  into  groupware  that  allow  the  designer  the  ability  to  input 
TLOs  and  ELOs  and  then  link  those  objectives  to  the  proper  task  or  sub-steps.  This  type  of 
organization  may  also  point  out  which  combination  of  tasks  is  being  over-taught  or  which  tasks 
could  be  combined.  It  is  the  electronic  organization  of  this  data  that  can  provide  a detailed  report 
of  task  data  associated  with  objectives  that  may  otherwise  be  overlooked. 

• Media  selection  algorithm  is  based  on  elements  such  as  cognitive  type  (from  the  simplest  recall  of 
facts  to  higher  levels  of  mental  processing,  such  as  analysis  of  data),  working  environment  under 
which  the  task  must  be  performed  for  training,  tied  to  the  learning  objective  (hence  performance 
level)  to  be  achieved. 

The  following  types  of  media  are  being  investigated  for  consideration  in  the  development  options 
for  training.  Some  are  legacy  (in  use  today)  while  others  are  more  advanced  awaiting  the  technology  for 
implementation: 

• Interactive  Multimedia  Instruction  (IMI)  - most  future  delivery  methods  are  via  the  web  and 
should  contain  various  interactivity  levels,  leaving  behind  the  “page  turner”  application.  IMI 
delivers  structured  content  while  tracking  learning  objectives. 

• Part  Task  Trainer  (PTT)  - use  of  computer  generated  animations  (usually  on  a desk- top 
computer),  or,  simulator  or  limited  simulator  training  that  enable  interactions  to  replicate  hands-on 
exercises.  For  example,  operation  of  a communications  radio  within  the  cockpit  of  an  aircraft. 

• Full  Task  Trainer  (FTT)  - simulated  equipment  closely  replicating  real-life  task  performance.  For 
example,  a flight  simulator  replicating  the  entire  cockpit. 

• Live  Simulations  - real  entities  (people)  are  operating  in  a real  environment  using  real  equipment. 
For  example,  an  aircraft  may  be  on  a mission  to  hit  a target  but  using  fake  munition. 

• Virtual  Simulations  - real  and  computer  generated  entities  are  present  and  interact  within  the 
environment.  You  may  be  logged  on  as  a particular  entity  (person)  in  a simulated  environment. 
Your  simulated  movements  are  tracked  against  computer  generated  entities  (which  could  be  the 
enemy). 

• Constructive  Simulations  - all  entities  are  computer  generated.  In  this  case,  you  are  simulated  in  a 
simulated  environment  (which  may  be  a game  environment)  in  which  other  elements  are 
completely  simulated  as  well. 

• Interactive  Electronic  Technical  Manuals  (IETMs)  - text-based  procedural  processes  with  visual 
supplements.  Interactive  IETMs  entail  interactive  visuals  that  use  scalable  graphics,  for  example,  a 
person  could  click  on  the  outer  shell  of  an  engine  and  watch  it  slowly  break  apart  to  show  internal 
detail.  Whereas,  future  IETM  development  involves  being  linked  to  equipment  and/or 
maintenance  network  and  integrates  diagnostics. 

As  a final  note,  each  project  may  require  a backup  of  communication  data  and  support 
documentation  for  decisions  made  throughout  the  project.  Using  such  tools  as  groupware  can  simplify  the 
reporting  of  this  information  to  the  customer.  These  reports  can  also  include  why  a certain  learning 
objective  was  developed  and/or  why  a specific  medium  was  chosen  over  another.  The  key  here  is  that  the 
groupware  ties  in  all  data  no  matter  what  the  use  within  the  project. 

Conclusion 

Communication  is  the  key  to  the  successful  completion  of  a TA.  We  should  not  work  within  a 
vacuum  and  then  distribute  the  information  upon  completion,  but  instead  information  should  be  shared 
along  the  entire  process  with  all  participants.  This  is  where  the  need  for  automated  collaboration  comes  in 
to  play.  The  following  summarizes  some  of  the  lessons  learned  from  our  experiences  in  developing, 
integrating,  using  and  managing  groupware  within  TA  efforts. 

• The  biggest  hurdle  in  implementing  groupware  is  convincing  the  participants  to  use  it.  Ensure  the 
participants  clearly  understand  the  benefits  in  using  the  groupware.  In  other  words,  begin 
recruitment  early. 

• Proper  training  is  required  to  make  participants  comfortable  in  using  it.  Insufficient  training  leads 
to  misuse  or  misunderstanding  of  the  groupware,  which  leads  to  its  non-use. 

• More  than  likely,  a combination  of  both  shrink-wrap  and  custom  groupware  may  have  to  be  used. 
Using  all  shrink-wrap  typically  doesn’t  fit  into  the  TA  effort’s  processes  unless  the  processes  are 
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then  changed  to  fit  the  existing  groupware,  while  an  all  custom  solution  is  typically  costly  and 
may  require  too  much  advanced  time  to  be  developed. 

• The  groupware  must  be  seamlessly  used  within  the  TA  process.  The  focus  of  the  end-user  should 
be  placed  on  the  process,  not  the  groupware.  Therefore,  if  possible,  it  is  best  to  choose 
technologies  that  fit  the  processes  in  place. 

• The  communication  styles  discussed,  synchronous,  asynchronous,  automated/pre-programmed, 
and  one-way  have  all  been  typically  used  in  some  capacity,  especially  with  large-scale  TA  efforts. 
Do  not  force-fit  these  styles  into  the  TA  effort  in  question,  but  instead,  only  use  those,  which  seem 
to  work  naturally  within  the  TA  effort. 

• Developing  a more  robust  groupware  application  to  handle  not  only  the  TA  but  the  segments  of 
design  and  development  should  lead  to  huge  cost  savings  through  automation  and  reuse  of  data. 

• Being  able  to  follow  a path  of  decision  points  that  are  electronically  captured,  be  it  email  strings  or 
reasons  behind  medium  selection,  can  be  important  to  the  wrap-up  of  a project. 
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The  Educational  Technology  Implementation  Plan  (ETip):  Why,  What, 

and  How? 

Julie  Reinhart 

The  University  of  North  Carolina  at  Greensboro 

Joseph  Slowinski 
The  New  Dimensions  School 

Why  Use  the  ETip  Model? 

The  ETip  model  grew  out  of  the  need  to  move  away  from  previous  technology  integration  models  that  have 
conceptualized  technology  integration  in  a narrow  manner.  For  instance,  educational  institutions  in  the  past 
would  simply  provide  Power  Point  training  on  a yearly  basis.  Professional  development  practices  should 
model  the  social  learning  practices  that  schools  want  their  teachers  to  utilize  in  the  classroom. 

Schools  need  to  have  a clear  vision  for  how  technology  is  going  to  be  used  in  their  environment.  This 
vision  needs  to  be  aligned  with  the  school  environments’  vision  for  quality  learning  and  instructional 
practices.  Additionally,  change  agents  need  to  understand  that  teachers’  use  of  technology  evolves  over 
time.  There  is  a developmental  continuum  that  teachers  tend  to  progress  through  as  they  begin  to  use 
technology  in  educational  environments  (Sherry,  L.,  Billig,  S.,  Tavalin,  F.  and  Gibson,  D.,  2000). 
Professional  development  should  vary  depending  on  the  needs  of  the  majority  of  participants  as  well  as 
unique  situated  needs  of  individuals.  In  order  to  do  this,  careful  planning  and  a solid  understanding  of  what 
technology  integration  is  and  how  technology  is  diffused  into  an  educational  environment  is  needed.  Thus, 
we  developed  a model  that  pulls  research  from  the  fields  of  social  learning,  change  management, 
innovation  diffusion,  motivation,  teacher  technology  adaptation  growth  processes,  and  the  best  practices  of 
professional  development. 


What  is  the  ETip  Model? 


Level  1 

Level  II 

Level  III 

Level  IV 

Level  V 

Educational 

Environment's 

Goal  @ Level  1 : 

Goal  @ Level  2: 

Goal  @ Level3: 

Goal  @ Level  4: 

Ultimate  Goal: 

Vision 

Members  of  the 

educational 

community 

become 

technology 

users. 

Technology  is 
utilized  to  assist 
with  classroom 
productivity. 

Curriculum  is 
enhanced  by 
technology. 

Students  construct 
knowledge  utilizing 
a wide  range  of 
tools  depending  on 
the  subject  matter. 
Technology  is  one 
of  the  tools  that 
students  may 
utilize. 

Students  and 
Teachers  use 
technology  as 
tools  for 
communication, 
collaboration,  and 
analysis. 

Teacher 

Target 

Personal 

Productivity: 

Teacher  uses 
technology  for 
personal  and 
task 

management 

concerns. 

Professional 

Productivity: 

Teacher  uses 
technology  in 
their  classroom 
for  professional 
task 

management 

concerns. 

Professional 

Enrichment: 

Teacher  builds  a 
clear  relationship 
with  technology 
and  curriculum. 

Paradigm  Shift: 

Teacher  changes 
the  manner  in 
which  they  teach 
and  assess.  The 
use  of  technology 
becomes  invisible. 

Visionary: 

Teacher  shares 
experiences  based 
on  own  research 
and  develops 
improvements  in 
practice. 

Role  of 

Teacher  in  PD 
Process 

Learner: 

teacher  spends 
time  learning 
how  to  use  the 

Adopter: 

teacher  adopts 
their  personal 
use  of 

Adapter:  teacher 
uses  the 
technology  for 
instruction  rather 

Designer:  teacher 
improves  the  use 
of  technology  for 
instruction  to  more 

Leader:  teacher 
trains  other 
teachers  on  the 
area  of  technology 
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technology. 

technology  for 

than  in  support  of 

closely  match  the 

integration  in 

professional 

productivity. 

instruction. 

students'  needs. 

which  they  excel. 

Pedagogical 
focus  of  the 
teacher 

Instructionism — 
Transmission 

Learning  by 
teacher  driven 
projects.  Student 
as  static  learner. 

Constructivism- 

Constructionism: 

Learning  by 
Making.  Student 
as  apprentice. 

Participatory 

Constructionism: 

Group  learning  in 
authentic  context 
and  situated 
learning.  Student 
as  creator. 

PD  Bridge 

WebQuest  Model 

ICON  Model , with 
Big  6 and  other 
information  literacy 
models 

Virtual 
Collaborati 
on  Support 

PD  Product 

Individual 

Support 

Enrich 

Support 

Student-Centered 

Growth: 

Instruction: 

Instruction: 

Development  and 

Learnina: 

Implementation: 

• Workshops, 

• Lunch  Bytes, 

• Mentor, 

• On-line  Just- 
in-time  (JIT), 

• Tutorials 
(online  & paper- 
based) 

• Workshops, 

• Lunch  Bytes, 

• Mentor, 

• On-line  JIT, 

• Resource 
search  engine, 

• Tutorials 
(online  & paper- 
based) 

• Mentor, 

• Modeling, 

• Lunch  Bytes, 

• On-line  JIT, 

• Resource  search 
engine 

• Virtual  Mentor, 

• On-line  JIT, 

• Resource  search 
engine 

• Virtual  collegial 
collaboration, 

• Resource 
search  engine. 

Role  of 
Change 
Agents 

It  is  important 
to  note  that 
this  section 
provides 
general 
guidelines. 

The 

leadership  in 
the  school 
should  decide 
the  appropriate 
approach  for 
moving 
teachers 
through  the 
levels  of 
technology 
implementatio 
n based  on  the 
teachers' 
needs  and 
school 
resources. 

Technology 
Facilitator  (TF): 

Provide 

workshops  and 
In-building  tech 
assistance. 

Also,  create  a 
technology  help 
line.  Begin 
building  an  on- 
line just-in-time 
repository  on  the 
schools  website 
that  teachers 
can  easily 
access.  Also, 
post  on  the 
Website  tutorials 
for  the  common 
software  that  the 
school  owns. 

Administration 

(Admin): 

Require  email 
for  school-based 
communication 
and  provide 
email  accounts 
to  teachers. 

Also,  implement 
the  use  of  a 
school-based 

TF:  Continue 
providing 
workshops. 
Target  train 
tech  savvy 
teachers  in 
order  to  enable 
the  teachers  to 
work  with  other 
teachers  in  the 
school,  continue 
the  use  of  the 
help  line,  add  to 
the  online 
resources. 

Admin:  Provide 
access  to 
equipment  in  the 
classroom  and 
support  after 
school 
workshops. 
Encourage  use 
of  technology. 
Administration 
should  also 
provide  time  for 
PD  and 

promotes/encou 
rages  teachers 
to  particpate. 
Teachers  should 

TF:  Move  away 
from  formal 
workshops 
towards  modeling 
and  mentoring. 
Continue  to  utilize 
the  assistance  of 
tech  savvy 
teachers.  Also, 
utilize  students  as 
informal  tech 
assistants. 
Continue 
developing 
technology-related 
online  resources 
for  teachers. 

School  Library 
Media  Specialist 
(SLMS):  Mentor 
and  Model  the  use 
of  information 
technology  for 
research  through 
information  literacy 
collaborations  with 
teachers.  Also, 
Alternate  Lunch 
Byte  sessions  with 
the  TF  and 
collaborate  with 
the  TF  to  create  a 

TF:  Utilize  virtual 
mentoring  rather 
than  face-to-face 
modeling  and 
mentoring. 
Continue  utilizing 
the  assistance  of 
tech  savvy 
teachers,  and 
students  as 
informal  tech 
assistants. 

SLMS:  Utilize 
virtual  mentoring 
for  information 
literacy 

collaborations  with 
teachers. 

Admin:  Continue 
with  what  was 
done  at  level  3 and 
if  possible  reward 
people  monetarily 
and/or  with 
equipment  for 
progressing  to 
these  student- 
centered  levels  on 
a daily  basis. 

TF  and  SLMS: 

Work 

collaboratively  with 
teachers  and 
students  in  group 
learning 
experiences. 
Continue  utilizing 
the  virtual 
environment  that 
you  used  to 
mentor  teachers 
but  at  this  stage 
the  interactions 
should  be  more 
collaborative. 

Also,  introduce 
new  and 
innovative 
technologies  and 
resources  as  they 
become  available. 

Admin:  Allow 
time  for  teachers 
to  collaborate. 

Also,  provide 
budget  based 
incentive 

programs  shiftring 
from  capital  to 
operating  through 
leasing. 
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Motivation 


Website.  Lobby 
the  school  board 
to  make  email 
an  official  form 
of 

communication 
and  eliminate 
paper-based 
daily  bulletins. 


Success: 

Create  an 
intrinsically 
motivating  tech 
project  that 
utilizes  process 
goals  rather 
than  product 
goals.  The 
subject  matter 
should  be 
grounded  in 
what  the  teacher 
does.  Provide 
follow-up  PD. 


receive  CEUs 
upon  completion 
of  a product. 


Success:  Use 

process  goals 
with  projects 
that  use 
technology  to 
automate 
professional 
tasks.  The 
subject  matter 
should  be 
grounded  in 
what  the  teacher 
does.  Provide 
follow-up  to  PD. 


resource  search 
engine. 

Admin: 

Administration 
promotes  and 
encourages 
project  -based 
CEUs  for  teacher 
PD.  Teachers 
receive  CEUs 
once  project  is 
complete. 
Administration 
also  encourages 
curriculum  specific 
project-based  PD. 

Public  Praise: 

Disseminate 
praise  for 
exemplary  student 
work.  Focus  on 
the  instructional 
method  not  the 
technology. 
Provide  follow-up 
to  PD. 


Public  Praise: 

Disseminate 
praise  for 
exemplary  project. 
Ed  Tech 
newsletters  or 
show  case  in  Tech 
Brief.  Focus  on 
the  instructional 
method  not  the 
technology. 


Rewards: 

ncentive  for 
facilitation 
opportunities; 
Showcase 
exemplary 
practices.  Ed  tech 
roundtables  during 
teacher  workdays. 


Examples 


Teacher  uses  E- 
mail  to 

communicate 
with  friends  and 
family. 


Teacher  uses 
DreamWeaver, 
to  create  a basic 
Webpage  to 
post  class  news, 
links  and 
homework 
assignments. 


Teacher  adopts  a 
WebQuest. 


Best  Practices: 
China  WebQuest 


Teacher  creates 
their  own  web 
quest  and 
technical  tools  are 
in  the  hands  of  the 
students.  Teacher 
also  collaborates 
with  colleagues. 

Best  Practices: 
Geometry  in  the 
Real  World 


Teacher  engages 
in  virtual 

collaboration  with 
a scientist  to  solve 
a local  authentic 
problem. 

Best  Practice:  Oral 


History  project 


Figure  1 : Educational  Technology  Implementation  Plan  (ETip) 

The  ETip  model  extends  the  work  of  Sherry,  L.,  Billig,  S.,  Tavalin,  F.  and  Gibson,  D.  (2000)  and 
their  Technology  Adoption  Model.  Sherry  et  al’s  model  was  based  on  the  Hall  and  Hord’s  Concems-Based 
Adoption  Model  (CBAM)  and  Everitt  Rogers,  Diffusions  of  Innovations  (1995).  The  failure  to  consider 
human  affect  and  perceived  value  tends  to  lead  to  failure  in  technology  integration.  This  viewpoint  parallels 
Hall  and  Hordes’  CBAM.  Hall  and  Hord’s  CBAM  model  is  an  appropriate  model  for  technology 
integration  due  to  the  emphasis  on  the  human  and  social  side  of  adopting  an  innovation.  It  is  the  human 
element  that  moves  the  technology  through  the  implementation  process,  this  is  why  we  added  a 
motivational  category  to  aid  the  school’s  change  agent  in  motivating  teachers  to  utilize  and  eventually 
integrate  technology  into  the  curriculum. 

The  motivational  category  is  based  on  Keller’s  (1983)  ARCs  model.  Keller’s  ARCS  Model  of 
Motivation  posits  that  if  the  learning  experience  attracts  the  learner’ s attention,  has  relevance  to  the  learner, 
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allows  the  learner  to  build  confidence  in  their  abilities  to  succeed  with  the  task  and  provides  a satisfying 
experience  then  the  learners  will  be  more  likely  expend  the  effort  with  the  learning  experience. 

The  ETip  model  also  incorporates  what  the  authors  refer  to  as  “professional  development  bridges” 
to  aide  teachers  with  progressing  through  the  levels  of  the  ETip  model.  It  has  been  our  experience  that 
often  times  teachers’  instructional  practices  with  technology  tend  to  plateau  at  a specific  level.  Therefore 
we  include  in  the  ETip  model  research-based  techniques  that  can  aide  in  moving  teachers  to  the  next  level. 
These  bridges  are  based  on  social  learning  principles,  our  own  research  and  experiences,  Bernie  Dodge's 
work  (1995,  2001),  and  the  work  of  Black  and  McClintock  (1995).  The  bridges  along  with  recommended 
professional  development  strategies,  that  are  based  on  best  practices  in  professional  development  (Rogers, 
2001),  provide  the  tools  that  change  agents  can  use  as  a bridge  from  one  level  to  the  next. 

How  is  the  ETip  Model  Used? 

In  order  to  use  the  ETip  model,  schools  need  to  first  determine  their  vision  for  technology  use  in 
their  school.  This  vision  needs  to  be  a shared  vision  that  the  entire  school  community  buys  into.  Then  the 
school  needs  to  determine  how  the  teachers  in  their  environment  are  currently  using  technology  in  the 
classroom.  Schools  can  either  make  their  own  inventory  or  utilize  a ready-made  inventory  such  as  the 
technology  use  instruments  that  Moersch  (2002)  evaluated. 

Once  the  school  has  a strong  understanding  of  how  the  teachers  are  using  the  model,  they  can  plan 
professional  development  activities  based  on  the  level  that  the  majority  of  the  teachers  are  at  on  the  ETip 
model.  If  there  are  small  clusters  of  teachers  at  other  levels  then  unique  professional  development  offerings 
can  be  developed  based  on  the  model  to  meet  the  specific  needs  of  the  teacher. 

The  E-TIP  model  builds  on  what  we  have  learned  about  social  learning,  change  management, 
diffusion  of  innovation,  teachers’  development  of  technology  utilization,  and  best  practices  in  professional 
development.  To  be  successful  in  the  development  of  a technology  professional  development  plan,  a 
district  must  first  recognize  that  quality  learning  is  made  possible  through  technology.  Then  school  districts 
can  use  the  ETip  model  to  further  the  use  of  technology  in  classroom  practices. 
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The  Problem 

According  to  Thomas  (2003)  instructional  designers,  instructional  technologists  as  well  as 
educators  in  general,  face  two  vexing  questions.  How  do  we  improve  our  instructional  materials  in  general? 
And  how  can  learner  perspective  or  culture  be  included  in  our  designs  to  make  them  more  effective  and 
relevant.  In  current  teacher  education  contexts  the  increasing  pressure  to  “prepare  preservice  teachers 
theoretically  and  practically  to  teach  culturally,  linguistically,  and  learning  diverse  students”  (Harris  et  al 
2005)  is  central  to  these  design  related  questions.  Allowing  preservice  teachers  to  become  designers  of 
digital  stories  or  life  centered  products  using  advanced  hypermedia  with  multimedia  authoring  software 
provides  an  important  response  to  these  critical  questions. 

The  purpose  of  this  paper  is  to  discuss  the  manner  in  which  advanced  hypermedia  and  digital  story 
telling  were  used  to  engender  a learning  by  design  environment  for  teacher  education  students.  In 
combination  the  discussion  will  also  address  the  value  of  such  complex  multimedia  software  tools  for 
making  instructional  materials  more  learner  centered  and  commensurate  with  the  students’  perspective. 

Theoretical  Framework 

Educational  theory  and  research  indicate  vast  potentials  for  increased  student  learning  by  adopting 
a “learner  as  designer”  approach  (Jonassen  2000,  Liu  1998;  Reeves,  1998;  Erickson  & Wilhelm,  1996; 
Beichner,  1994;  Lehrer,  Erickson,  & Connell,  1994;  Spoehr,  1994.).  Contemporary  instructional 
technologists  have  suggested  that  “a  learning  by  design”  (Jonassen,  1998,  1999;  Jonassen  et  al,  2000; 
Resnick,  2003)  approach  is  an  effective  way  to  enhance  content  knowledge.  Based  upon  his  own 
experiences  in  developing  Instructional  materials  Jonassen  has  argued  that  “The  people  who  learn  the  most 
from  the  design  and  development  of  instructional  materials  are  the  designers. ’’(undated).  Diffusing  design 
principles  and  instructional  methods  to  pre- service  teachers  is  an  effective  way  for  us  to  improve 
instructional  technology  programs.  This  approach  then,  should  in  part,  characterize  how  teacher  education 
technology  courses  are  designed. 

In  this  regard  “learning  by  designing”  or  “learning  by  authoring”  has  been  found  to  have  a number 
of  positive  effects  on  learning  in  different  settings.  Primarily  this  approach  allows  learners  to  become 
producers  of  information  than  simply  consumers.  For  such  learners  technology  becomes  more  than  a tool 
for  reproducing  or  giving  information  (Jonassen  2000;  Nicaise  & Crane,  1999). 

Traditional  approaches  to  learning  by  design  have  centered  on  the  use  of  computer  based  objects 
for  problem  solving  and  process  thinking  (Resnick,  1998;  Resnick  et  al  1991).  These  approaches  are 
usually  centered  in  the  mathematics  as  well  as  science  classroom  and  involve  the  use  of  computer 
programming  such  as  LOGO.  These  approaches  though  effective,  are  often  divorced  from  the  experiences 
of  most  teachers.  The  use  of  LEGO/Logo  programmable  blocks  or  other  types  of  manipulatives  may  not  be 
patently  obvious  in  language  arts,  social  studies  or  other  none  mathematical  or  science  based  setting.  In  the 
interest  of  meaningful  or  authentic  learning  this  approach  to  learning  by  design  has  to  be  repurposed  for  the 
general  population  of  teachers.  The  use  of  hypermedia  or  multimedia  design  is  an  important  part  of  this 
endeavor.  There  is  in  fact  an  emerging  body  of  literature  which  supports  the  use  of  hypermedia  (hypertext 
and  multimedia)  design  in  educational  contexts  (Jeehon,  2004;  Liu  et  al  1997). 

The  use  of  digital  storytelling  in  this  context  provides  a basis  for  infusing  learner  voice  or 
perspective  in  this  design  activity.  Digital  storytelling  is  an  emerging  instructional  practice  for  broadening 
learning  in  many  instructional  contexts  (Bull  and  Kajder,  2004).  To  be  sure,  constructivist  theorists  have 
long  called  for  the  use  of  stories  as  a way  of  infusing  stories  for  instructional  supports  for  problem  solving 
(Jonassen  & Hernandez-Serano,  2002).  The  central  premise  of  digital  storytelling  is  that  it  “allows  the 
author  to  experience  the  power  of  personal  expression”  (Bull  & Kajder,  2004.  pp.48).  This  personal 


449 


connection  provides  a basis  for  problem  solving  and  creative  thinking.  Both  of  these  are  constructs  which 
are  important  to  a learner’s  “meaning-making”. 

Learning  by  design  provides  a compelling  research-centered  way  for  teachers  becoming  designers 
of  instructional  materials  on  the  one  hand.  On  the  other  digital  storytelling  provides  a basis  for  integrating 
learner  centered  experiences  into  the  instructional  setting.  As  learners  develop  digital  stories  they  actively 
engage  in  selecting  a narrative  or  an  experience  that  emerges  from  their  own  socio-cultural  context.  These 
are  usually  by  default  complex  emotional  experiences  that  students  may  not  usually  bring  into  a technology 
education  context.  This  provides  the  real-world  experience  that  allows  learners  to  remain  engaged  and 
motivated  throughout  the  project. 

Multimedia  is  defined  as  by  Peck  (1999)  as  a blending  of  two  or  more  types  of  media  “to  create  a 
sequence  of  events  that  will  communicate  an  idea  visually  with  both  sound  and  visual  support”  (Peck  cited 
in  Speaker,  2004  pp.242).  Hypermedia  on  the  other  hand  refers  to  linked  media  or  interactive  media. 
Hypermedia  is  rooted  in  Bush’s  (1945)  concept  as  articulated  in  his  widely  known  article  “As  We  May 
Think.”  Recently  there  has  been  considerable  fusion  and  interchange  of  these  concepts.  For  our  purposes 
we  use  the  words  hyper/multimedia  to  describe  those  elements  that  reflect  both  interactive  linked  media  as 
well  as  multiple  types  of  media. 

A review  of  the  literature  reveals  positive  effects  with  student  created  multimedia  projects. 
Beichner  (1994)  in  a study  of  middle  school-age  students  found  that  learners  where  highly  motivated  and 
increased  time  on  task  while  developing  multimedia.  He  further  found  that  while  engaging  in  this  process 
learners  become  more  concerned  with  the  accuracy  of  the  information  in  their  , products,  learners 
willingly  became  self-regulated  and  decisive  about  content  as  well  as  editing  processes  and  learners 
engaged  in  deep  research  on  different  instructional  materials  in  order  to  gain  information  about  their 
products.  Lehrer  et  al  (1994)  supported  much  of  Beichner’ s findings. 

Liu  and  Rutledge  (1997)  investigated  the  learning  by  designing  approach  motivated  at-risk  high 
school  students  in  designing  multimedia.  They  found  that  authoring  multimedia  gave  students  high  levels 
of  intrinsic  motivation.  Students  also  indicated  that  learning  by  design  allowed  them  to  interact  with  design 
skills  such  as  planning,  presenting,  collaborating,  mental  effort  and  interest  (Jeeheon,  2004). 

Studies  among  older  students  reveal  that  through  learning  by  designing  of  hyper/multimedia 
learners  are  able  to  think  about  concepts  in  a more  sophisticated  way  (Oughton  and  Reed  1998).  Oughton 
in  a study  of  knowledge  acquisition,  problem  solving  and  general  design  skills  of  15  high  school  students, 
found  inter  alia  that  “the  use  of  hypermedia  to  provide  opportunities  for  students  to  construct  knowledge 
can  be  a means  of  providing  meaningful  learning  activities”  (p.363). 

In  a very  revealing  study  of  graduate  students’  development  of  hypermedia  with  concept  mapping, 
Oughton  and  Reed  (2000)  found  that  hypermedia  development  was  influenced  by  learning  styles.  They 
found  that  student  hypermedia  concept  maps  were  more  developed  if  according  to  the  Kolb’s  Learning 
Style  Inventory  they  were  Divergers  or  Assimilators.  They  explain  that: 

Di vergers  have  the  ability  to  see  many  perspectives,  gather  wide-ranging  information,  and 
generate  many  ideas  (Jonassen  & Grabowski,  1993).  Assimilators  have  the  ability  to  assimilate 
wide-ranging  ideas  and  create  multiple  perspectives  (Jonassen  & Grabowski),  characteristics  that 
the  other  two  learning  styles  do  not  possess.  It  may  be  that  these  strengths  of  Assimilators  and 
Divergers  enabled  them  to  be  more  productive  when  generating  their  concept  maps  (cited  in 
Oughton  & Reed,  2000). 

In  providing  effective  cognitive  apprenticeships  for  preservice  teachers  we  can  benefit  from 
adopting  the  development  of  hyper/multimedia  with  a learning  by  designing  approach.  This  allows 
preservice  teachers  to  interact  with  complex  skills  as  well  as  cognitive  processes  (e.g.  planning,  presenting, 
collaborating,  critical  thinking  and  interest)  which  they  will  apply  daily  in  the  context  of  their  career. 
Additionally,  this  approach  allows  preservice  teachers  to  learn  to  use  technology  in  a way  that  they  can 
later  apply  in  their  own  teaching  settings. 

In  implementing  a learning  by  design  environment  for  creating  hyper/multimedia  production, 
teacher  educators  need  software  tools  that  allow  preservice  teachers  to  become  actively  engaged  in 
complex  cognitive  processes.  Considering  what  is  known  about  the  efficacy  of  this  approach,  such  software 
tools  should  be  selected  based  upon  their  ability  to  engender  the  complexity  and  authenticity  that  is 
required  for  the  effective  development  of  digital  stories.  These  tools  should  be  selected  for  their 
effectiveness,  instructional  adequacy,  interface  design  & navigation,  program  adequacy  and  documentation 
(Gibbs  et  al,  2001). 
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Advanced  Hypermedia 

There  are  multiple  software  packages  that  we  can  use  to  involve  preservice  teachers  in  learning  by 
design  when  developing  digital  stories.  Generally  speaking  the  best  types  of  tools  to  use  are  multimedia 
tools.  We  argue  that  Type  5 multimedia  tools  are  the  most  effective.  Type  5 tools  are  synchronized  video 
creation  with  hyper/multimedia  tools.  These  tools  have  recently  found  maturity  and  are  available  in 
commercial  versions  as  Microsoft  PowerPoint  Producer,  Articulate  Presenter  and  Macromedia  Captivate. 
All  allow  for  the  creation  of  interactive  media  that  accept  user  input  e.g.  hyperlinks  from  within  the  project 
to  other  sites,  branching  via  a table  of  contents  and  the  synchronization  of  voice,  video  & slides  for 
publishing  online.  Articulate  Presenter  allows  for  the  saving  of  notes,  viewing  of  references,  adding  of 
bookmarks  and  annotations  within  the  project,  s allow  for  the  saving  of  notes,  branching  through  a table  of 
contents  and  use  multiple  viewing  avenues.  These  include  synchronized  sound,  video  and  pictures.  There 
real  power  lay  in  their  ability  to  engage  students  in  creating  with  multiple  types  of  complex  media 
including  synchronized  video,  sound,  and  slideshows. 

Traditional  multimedia  tools  have  been  classified  by  Roblyer  (2004)  on  a four-tiered  typology. 
These  categories  reflect  the  level  of  complexity  of  mental  processes  that  each  set  of  software  tools  activate 
during  the  design  or  application  process.  These  tools  have  traditionally  been  used  in  hyper/multimedia 
learning  by  design  settings.  Roblyer  suggests  that  Type  1 authoring  systems  are  primarily  Presentation 
Software;  Type  2 is  Video  Production  and  Editing  Systems;  Type  3 is  Hypermedia  Software  and  Type  4 is 
Virtual  Reality  and  Other  immersion  systems 

Presentation  software  such  as  slideshow  presentation  tools  e.g.  Microsoft  PowerPoint,  Star  Office 
Impress,  and  other  linear  tools  are  used  to  create  as  well  as  engage  in  descriptive  or  content  presentations 
projected  off  a screen  (or  viewed  on  a computer).  These  slideshow  presentations  usually  integrated  sound, 
pictures,  video  and  text.  This  approach  to  media  development  has  come  under  increasing  criticism.  Some 
argue  that  its  use  makes  complex  concepts  as  well  as  ideas  appear  too  simplistic  and  to  some  extent  bears 
some  responsibility  for  the  Columbia  shuttle  disaster  (Thompson,  2003).  Others  say  this  approach  reduces 
the  “quality  and  credibility”  of  communication  (Tufte,  2003a,  2003b).  Still  others  argue  that  this  approach 
to  communication  and  presentation  represents  a type  of  “hegemonic  visualism”  (Tietje  & Cresap,  2005) 
that  values  one  approach  to  presenting  and  communicating  over  others. 

Video  production  and  editing  systems  is  conceptualized  by  Roblyer  as  type  2 multimedia  tools.  In 
recent  times  Apple  and  Microsoft  have  begun  to  disseminate  free  versions  of  video  production  and  editing 
tools.  Apple  iMovie  and  Windows  Movie  maker  are  now  widely  distributed  through  packaging  with  the 
standard  operating  systems  of  these  companies.  This  has  led  to  the  widening  development  of  movie 
production  in  k-12  schools.  Apple  has  a web  based  project  devoted  to  these  types  of  productions  among  k- 
12  students  called  the  Apple  Learning  Interchange  (http://ali.apple.com/). 

These  tools  involve  the  creation  and  later  viewing  (or  publishing  on  the  www)  moving  video. 
These  tools  allow  for  easy  capturing  of  videos  from  a camera  or  VCR.  They  are  then  edited  by  cutting, 
pasting,  adding  still  pictures,  effects,  transitions  and  soundtracks.  These  videos  can  then  be  saved  back  to  a 
tape  or  processed  for  publishing  on  a website.  They  are  often  used  to  create  interesting  digital  stories  about 
student  life  in  school  or  to  present  content  from  lessons  or  fieldtrips. 

Research  on  the  cognitive  uses  of  video  production  and  editing  for  students  in  an  instructional 
setting  remains  sparse.  We  can  however  argue  that  as  with  other  types  of  multimedia  production,  video  has 
the  potential  to  increase  engagement  as  well  as  interest  and  motivation.  Students  interact  with  complex 
operations  such  as  sequencing,  research  around  topics  or  content,  organizing  and  selecting  while  using 
these  tools.  As  our  society  continues  to  rely  more  on  videos  video  production  will  invariably  help  students 
to  assess  how  visual  images  are  used  to  persuade,  communicate  biases  and  form  a basis  for  public 
education  or  influence  (Roblyer,  2004). 

Type  3 tools  are  those  tools  that  involve  the  use  of  Flash  cards  or  webpages  as  interactive 
hypermedia.  For  Roblyer,  Hypermedia  software  is  distinct  from  the  previous  tools  because  they  engage  the 
learner  in  non-linear  design  activities.  Such  tools  encompass  hypermedia  tools  such  as  Macromedia 
Authorware,  Director,  and  Flash  as  well  as  stack  or  card  software  tools  such  as  HyperStudio.  They  also 
include  web  authoring  tools  such  as  Microsoft  FrontPage,  Macromedia  Dreamweaver  and  non  commercial 
products  (freeware)  tools  such  as  NVU.  These  tools  allow  the  learner  to  create  html  type  web  pages  or 
animations  that  employ  branching,  linking,  and  multiple  access  points  to  information.  While  developing 
these  types  of  products  learners  are  constantly  engaged  in  organizing,  sequencing  and  navigating  through 
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information.  They  are  also  actively  engaged  in  assessing  their  product  since  they  recognize  that  others  will 
be  viewing  it  as  well  (Roblyer  2004). 

Virtual  reality  (VR)  authoring  systems  are  still  in  a nascent  stage  of  application  in  educational 
contexts.  VR  interaction  systems  such  as  games,  tutorials  and  instructional  tools  however  have  gained 
popularity  for  educational  use  ().  Roblyer  sees  these  tools  as  type  4 multimedia  tools.  They  involve  the  use 
of  multimedia  elements  that  immerse  the  learner  into  a realistic  context  where  the  learner  can  explore,  view 
how  systems  work  as  well  as  interact  and  engage  in  practice.  Commercial  products  such  as  encyclopedias 
(e.g.  Encarta)  and  online  courseware  (e.g.  River  Deep)  makes  extensive  use  of  these  types  of  systems.  They 
are  also  common  in  educational  games. 

Unlike  the  previous  systems  type  5 authoring  systems  allow  for  the  linking  of  audio,  moving 
video,  still  pictures,  slide  shows  and  html  pages.  This  multifaceted  environment  also  allows  for  the 
synchronization  of  sound  with  videos  so  that  viewable  activities  can  be  timed  and  harmonized.  Users  can 
also  interact  with  these  elements  by  clicking  on  links  that  branch  to  different  points  of  the  presentation 
rather  than  move  through  the  show  in  linear  fashion  as  in  traditional  slideshows. 

PowerPoint  producer  for  example  has  a complex  but  navigable  production  interface  (fig.  1).  As  the 
figure  below  shows,  this  software  tool  allows  the  user  to  interact  with  multimedia  elements  within  one 
interface.  This  captures  all  of  the  cognitive  elements  of  all  of  Roblyer’ s media  categories.  For  example,  the 
story  can  have  a multimedia  slideshow  with  hyperlinked  elements  on  the  slides  while  retaining  voice 
narration  that  is  then  synchronized  with  video  segments  throughout.  The  creator  can  include  a table  of 
contents  for  quick  viewing  as  well  as  other  elements  such  as  running  notes  for  users  to  follow  along  or  save 
for  further  review. 

These  authoring  systems  are  easily  associated  with  the  process  of  learning  by  designing.  They 
promote  high  levels  of  problem  solving;  they  promote  engagement,  require  hands  on  learner  participation 
and  allow  for  the  creation  of  real-world  artifacts.  Produced  elements  can  actually  be  published  to  a variety 
of  media  (World  Wide  Web,  CDROM,  and  computer)  and  in  a variety  of  formats  from  presentation  movies 
that  include  slideshows,  to  compelling  online  tutorials  that  act  as  showcases  of  student  knowledge. 

Methodology 

Advanced  multimedia  projects  created  by  34  students  in  an  undergraduate  applied  instructional 
technology  course  were  examined.  Most  students  developed  projects  in  dyads,  however  some  students 
developed  their  own  individual  projects.  A total  of  fourteen  projects  were  reviewed  using  Liu’s  (1998) 
categorization  about  implementing  a learning  by  designing  environment  as  a framework  for  analysis. 

The  Learning  Environment 

For  the  past  year  we  have  implemented  the  use  of  these  advanced  hyper/multimedia  authoring 
systems  with  34  students  in  an  introductory  instructional  technology  course  for  teacher  education  students. 
Students  were  placed  in  dyads  and  asked  to  create  a digital  story  project  using  PowerPoint  Producer  2003. 
The  design  framework  used  in  this  process  was  centered  on  Lehrer  and  his  colleagues  as  well  as  Liu’s  work 
on  learning  by  designing  with  multimedia  (Lehrer  1993;  Lehrer  et  al  1994  & Liu  1998),  this  involved  a four 
phase  approach  planning,  designing,  producing  and  evaluation  and  revision. 

Table  1.  Cognitive  Skills  Related  to  Learning  by  Design  with  Multimedia  Design  Model 


Phases  of  Design  Model 

Design  Component  & Activities 

Cognitive  Skills  Targeted* 

Planning 

Defining  the  Nature  of  the  Problem 

- Viewing  other  digital  stories 

- Instructor  led  tutorials, 
discussions  & demonstrations 
Problem  Decomposition 

- Process  diagrams  or  establishing 
timelines 

Posing  questions,  problems  or  challenges 
Dividing  tasks  with  collaborators/peers 
Determining  the  elements  (tasks)  of  the 
problem 

Analysis  and  conceptualization  of  solution 
Creating  a timeline 

Electronic  database  & document  searching 

Designing 

Finding  Information 
Developing  New  Information 
- Storyboarding,  concept  mapping, 
sketching,  outlining 
Selecting  Information 

Taking  notes  making  annotations 
Narrative  or  script  writing 
Deciding  on  process  and  structure 
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Producing 

- Group  discussions 
Organizing  Information 
Representing  Information 
Applying  multimedia  tools  in 
through  scaffolding  (PowerPoint, 
web  pages,  video  editing  then 
synchronization) 

Acquiring  graphics,  pictures  or 
sounds 

Applying  animations,  transitions 
and  varied  effects 

Use  of  Database  Tools,  Semantic  Mapping 
Segmenting  Video  & Sound 
Interweaving  Media,  Graphic  & Video 
Production 

Representing  what  is  known  via  different 
media 

Transmit  elements  to  diverse  media 
Managing  limitations  (time,  resources,  tools 
etc) 

Evaluating  & Revising 

Ongoing  self-  evaluation  of  the 
design 

Group  evaluation  of  the  design 
Instructor  evaluation  of  the  design 

Receiving  and  applying  feedback  (peers, 
instructor  & via  reflection) 

Going  public  via  demonstration/presentation 

* These  thinking  skills  are  those  cited  by  Liu  from  Carver  et  al  1992 


Results 

Students  generally  produced  projects  that  fell  into  three  categories,  a.)  Self-identity  or  culture 
based  projects  b.)  Professional  need  or  practice  projects  and  c.)  Digital  stories  of  dramatic  or  historical 
events  projects.  These  choices  were  interesting  in  and  of  themselves  for  they  clearly  reflect  the  desire  on 
the  part  of  students  to  represent  their  own  interests  in  the  design  of  multimedia  learning  materials.  Each 
type  of  project  that  was  created  in  some  senses  reflected  different  levels  of  interest  on  the  part  of  students 
but  involved  complex  design  tasks  nonetheless. 

Self-identity  projects  involved  the  creation  of  projects  that  reflected  the  cultural  experiences  of 
students.  In  figure  2 for  example  students  retold  the  story  of  “How  dogs  became  man’s  best  friend”  in 
Russian  and  English.  A video  of  students  retelling  the  story  and  wearing  Russian  traditional  dress  were 
juxtaposed  with  a slideshow  of  interesting  words  and  events.  Other  interesting  projects  included  the 
retelling  of  Maya  Angelou’s  And  Still  I Rise,  Dr.  Seues’  The  Sneetches  and  a Native  American  folktale  The 
Rough  Faced  Girl. 

Professional  practice  projects  were  based  on  the  creation  of  lesson  materials  and  classroom 
management  slideshows  for  students  and  parents  to  use.  Figure  3 depicts  an  open  house  production  that  a 
mathematics  teacher  created  as  an  introduction  to  the  school  and  classroom  environment.  Other  ideas 
included  lessons  and  tutorials  on  American  Sign  Language,  letters  of  the  Alphabet  and  book  summaries. 

Students  also  used  this  medium  for  creating  digital  stories  that  depicted  personal  experiences  or 
historical  events.  Students  drew  upon  the  civil  rights  movement.  Using  a wealth  of  resources  they  produced 
a rich  media  production  that  juxtaposed  images  of  the  civil  rights  movement  (project  called  the  Watsons  - 
based  on  the  book  the  Watson ’s  go  to  Birmingham)  with  voice  narration  as  well  as  inserting  speeches  from 
that  era  e.g.  Dr.  Martin  Luther  King’s  I have  a dream  (Fig.  4).  Other  projects  included  personal  experiences 
on  the  Tsunami  in  Sri  Lanka. 

The  finished  hyper/multimedia  is  published  to  a website  where  it  is  viewable  in  any  standard  web 
browser.  The  software  packages  automatically  generate  the  table  of  contents,  as  well  as  layout,  and  entry 
(or  exit)  of  synchronized  elements.  At  the  production  level  the  creator  simply  sequences  the  main  elements 
(audio,  video,  pictures  and  slideshow)  on  the  timeline  as  well  as  edits  the  table  of  contents. 

These  types  of  projects  allowed  students  to  be  centered  in  the  learning  context  and  provided 
motivation  for  them  to  be  actively  engaged.  As  is  commensurate  with  previous  findings  (Beichner,  1994) 
my  students  were  often  willing  to  put  in  longer  hours  in  the  lab  to  develop  projects  and  schedule  time 
outside  of  class  to  meet  in  production  teams  for  filming  video  footage  and  recording  voice  narrations.  They 
were  also  continuously  engaged  in  research  for  source  materials  (pictures,  music,  articles,  books)  that 
would  help  them  to  represent  their  content  better  using  the  media. 

Previous  studies  identified  that  while  developing  multimedia  learners  “internalize  various  design 
skills  (Liu  p.  388).”  In  these  studies  “students  reported  increases  in  mental  effort  and  involvement,  interest, 
planning,  collaboration,  and  individualization”  (Liu  p.  388).”  In  our  own  work  with  advanced  hypermedia 
we  observed  that  students  engaged  in  these  elements  as  well.  One  student  commented  that  this  approach 
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increased  their  collaborative  experiences  and  reflected  their  perspective.  She  states  a positive  likeness 
toward  this  approach  “because  I continue  to  get  experience  in  group  work,  and  getting  along  with  others.  It 
also  showed  how  each  of  us  had  a different  view  on  how  we  wanted  the  video  [digital  story]  and  the  type  of 
books  we  wanted.  But  digital  storytelling  lets  the  students  express  their  views.” 

Of  the  three  project  types  that  students  opted  to  create,  the  digital  story  was  found  to  be  more 
complex  and  more  media  rich.  Perhaps  because  of  the  number  of  digital  resources  available  on  the  world 
wide  web  surrounding  important  historical  events  these  projects  tended  to  be  more  sophisticated.  For 
example,  students  would  try  to  join  two  audio  clips  or  video  clips  related  to  the  event.  So  the  Watsons 
project  begins  with  the  I have  a dream  speech  but  then  goes  into  the  authors,  voice  narration.  Students 
would  also  conceptualize  different  scenarios  for  representing  an  event  that  was  sometimes  beyond  the 
limits  of  the  software.  E.g.  the  Tsunami  project  authors  wanted  to  show  an  animation  of  how  the  moving 
flood  waters  forever  altered  the  South  & East  coastlines  of  Sri  Lanka. 

The  most  interesting  result  of  this  process  at  least  from  the  standpoint  of  the  instructor  is  the  extent 
to  which  students  were  able  to  represent  the  complexity  in  their  own  thinking  using  this  tool.  One  student 
felt  that  the  complexity  of  the  media  by  fusing  video,  slideshow  and  voice  narrative  allowed  for  the 
organizing,  sequencing  and  publication  of  complex  thoughts  that  would  have  otherwise  been  difficult  to 
achieve.  In  synchronizing  this  with  a table  of  contents  and  various  hyperlinks  to  other  aspects  of  the  story, 
the  viewer  can  interact  with  this  complex  experience  that  would  have  been  impossible  through  the 
traditional  approaches  to  storytelling  (face-to-face)  as  well  as  traditional  approaches  to  moving  video  (basic 
video  production).  The  digital  storytelling  approach  allowed  students  to  draw  upon  rich  media  in  order  to 
transmit  complex  thoughts. 

As  is  commensurate  with  Oughton  and  Reed  (1998)  it  was  found  that  students  did  deeply  engage 
in  complex  cognition  and  problem  solving  as  they  interacted  with  by  developing  digital  stories  through  the 
use  advanced  multimedia.  Through  this  process  preservice  teachers  engaged  in  problem  solving  in  an  ill 
structured  learning  domain.  Preservice  teachers  learned  to  use  PowerPoint  Producer  not  simply  to  learn 
about  the  tool  but  they  also  learn  important  tasks  such  as  planning,  designing,  evaluating  as  important 
design  principles.  Students  appreciated  the  assessment  and  revision  aspect  because  it  allowed  them  to 
refine  their  products  as  one  student  stated.  This  approach  “gave  students  time  to  perfect  their  work,  and  fix 
mistakes  they  weren’t  suspected  to  run  into!” 

These  stories  played  an  important  role  in  infusing  the  learners  perspective  into  the  material 
designed.  The  use  of  digital  storytelling  allowed  the  preservice  teachers  to  engage  in  the  use  of  complex 
media  in  order  to  represent  complex  content.  Students  commented  on  the  fact  that  digital  storytelling  in  this 
way  could  be  carried  out  through  different  types  of  media.  Additionally,  they  recognized  that  whatever  is 
designed  inherently  carries  the  learners  perspective.  Student  comments  ranged  from  “Through  digital  story 
telling,  all  students  are  able  to  benefit  from  visual  and  audio  technologies  since  both  reinforce  learning.”  As 
well  as  including  comments  like,  “I  can  see  that  each  student  might  approach  the  story  differently  with 
various  reading  styles  and  pictures.  Also  different  takes  on  the  message  of  a story  can  come  out  in  this 
storytelling  technique  and  be  shared  with  other  students.” 

Another  important  result  was  the  fact  that  student  projects  reflected  nuances  toward  important 
skills  required  particularly  among  diverse  populations  such  as  ESOL  or  struggling  English  Language 
learners.  While  developing  the  scripts  and  narratives  for  the  production  preservice  teachers  engaged  in 
developing  lengthy  narratives.  Additionally,  the  slideshow  portions  made  extensive  use  of  text.  In  this 
regard  language  skills  were  brought  into  the  production  as  a central  tool  for  product  completion.  Some 
students  actually  chose  to  create  the  narrative  before  developing  the  storyboard  as  they  felt  that  this  was  a 
better  way  to  organize  their  thoughts. 

The  PowerPoint  Producer  software  tool  was  observed  to  be  effective  tools  for  preservice  teachers 
to  use.  Students  were  able  to  complete  highly  developed  rich  media  projects  within  the  context  of  the 
overall  semester  (one  summer  session  of  8 weeks  and  a regular  16  week  session)  along  with  other 
curricular  projects  as  well.  Students  did  not  display  great  difficulty  in  navigating  the  PowerPoint  Producer 
software.  In  an  informal  survey  students  expressed  that  the  digital  storytelling  project  allowed  them  to  see 
how  different  technologies  can  be  applied  on  comment  said  that,  “the  digital  storytelling  project  did  help 
me  to  realize  different  perspectives  and  different  uses  of  technology.” 

Implications 

The  use  of  advanced  multimedia  as  a software  tool  for  developing  digital  stories  is  compelling.  In 
large  part  these  tools  are  just  emerging  as  robust  and  applicable  to  the  educational  context.  In  education  in 


454 


general  and  in  our  own  case  in  particular  these  tools  remain  underused.  This  media  should  be  exploited 
especially  in  relation  to  digital  story  telling  as  the  digital  story  provides  the  complexity  for  the  media  to  be 
effectively  used. 

Using  this  type  of  software  tool  with  digital  storytelling  is  different  from  simple  slide  show  digital 
stories  or  even  traditional  moving  video.  These  traditional  types  interact  with  a single  set  of  source  files 
that  form  the  focal  point  of  the  production  i.e.  slideshow  or  video.  In  the  case  of  the  advanced  multimedia 
products  there  is  an  opportunity  to  select  any  of  three  main  media  types  as  the  focal  point.  This  means  that 
there  is  an  inherent  complexity  that  lends  itself  to  representation  of  complex  content. 

Advanced  hypermedia  of  this  type  5 category,  has  implicit  value  for  infusing  learning  by  design 
among  ESOL  students  or  other  students  with  emerging  language  skills.  The  use  of  the  narrative  script  and 
the  slideshow  form  important  scaffolds  to  getting  new  or  emerging  English  Language  learners  to  use  the 
language  openly  and  freely  with  what  is  known  to  be  an  engaging  approach.  Students  who  are  gifted  or  are 
highly  technologically  literate  will  also  find  intrinsic  value  in  using  this  approach  as  well.  Primarily  type  5 
hypermedia  with  digital  storytelling  engages  the  learner  because  of  the  pedagogical  value  not  because  of 
the  technical  or  aesthetic  factor.  Preservice  and  Inservice  teachers  that  utilize  this  approach  will  find  great 
value  in  it  for  giving  students  voice  and  perspective. 


Fig.  1:  Production  interface  of  PowerPoint  Producer 


Fig. 2:  English  & Russian  Folktale  (with  video  & voice) 
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OPEN  HOUSE 
Welcome  Parents! 

School  Policies 
Curriculum  Coals 
Class  Rules 

Opportunities  to  Cat  Involved 
Handouts 
Contact  Information 
Than*  you  for  attending! 


Fig.  3:  Open  house  classroom  production 


Fig. 4:  Civil  Rights  Digital  Story  in  PowerPoint  Producer  2003 
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A Systems  Approach  to  High-Quality  Online  Course  Development 

Guidelines 


Ruth  Rominger 
Brian  Rowlett 

Monterey  Institute  for  Technology  and  Education 

The  National  Repository  of  Online  Courses  (NROC)  project  supports  the  development  and 
distribution  of  high-quality  online  courses  to  a worldwide  audience.  The  goal  of  this  project  is  to  facilitate 
collaboration  among  a community  of  developers  to  create  a library  of  online  courses  that  are  available  to 
instructors  in  academic  institutions  everywhere.  Supported  by  The  William  and  Flora  Hewlett  Foundation, 
the  library  launched  this  summer  offering  courses  for  high  school,  Advanced  Placement©,  and  higher 
education. 

A Sustainable  Business  Model 

NROC  is  being  built  upon  an  innovative  business  model  that  is  prototyping  a complete  online 
learning  “value  chain.”  including  editorial  and  technical  guidelines,  a collaborative  development  network, 
and  options  for  flexible  delivery  and  use  by  instructors.  The  model  consists  of  pedagogical  and  technical 
guidelines  for  course  development,  a collaborative  development  network,  options  for  flexible  delivery, 
barter,  and  licensing,  customization,  and  facilitation  by  instructors. 

Sustainable  Online  Course  Value  Chain 


Figure  1 
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Collaborative  Development 

As  the  online  education  field  takes  off,  tens  millions  of  dollars  are  being  spent  on  developing 
courses  at  the  instructor  and  class  level.  It  is  safe  to  say  that  tens  of  thousands  of  courses  are  being  made 
throughout  the  world.  Yet  when  we  survey  the  courses  that  are  in  use  and  in  development,  a tiny  number  of 
the  courses  employ  innovative  learning  theory,  high  production  value,  or  interoperable  technical 
specifications.  With  this  as  our  landscape,  NROC  has  built  a collaborative  development  model  that  allows 
course  developers  to  share  guidelines  and  expertise,  advise  on  design  and  implementation,  seek  joint 
funding  for  collaborative  development,  match-make  institutions  with  the  same  needs  to  leverage  existing 
resources  into  sharing  curriculum  development. 
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Global  Distribution 

NROC  partners  with  academic  institutions,  publishers,  teaching  organizations,  state  and  federal 
agencies,  international  distributors,  social  benefit  programs  and  others  to  create  a global  distribution 
network  to  provide  courses  to  student,  teacher,  and  the  general  public  at  little  or  no  cost.  There  are  a 
number  of  ways  in  which  courses  may  be  licensed  for  use,  including  barter  (trading  for  contributing 
resources  to  the  library),  low-cost  library  licenses  and  open  access  licenses. 

NROC  Guidelines 

Our  guidelines  provide  a comprehensive  framework  and  set  of  principles  that  encompass  what  is 
known  about  effective  online  education,  that  have  the  flexibility  to  evolve  as  the  marketplace  of 
technologies,  collaboration,  investment,  and  learner  and  teacher  experiences  shift.  The  Guidelines  are 
designed  to  help  bridge  the  gap  between  what  research  tells  us  about  learning,  the  current  state  of  the  online 
learning  marketplace,  and  the  existing  technologies.  The  long-term  goal  of  National  Repository  of  Online 
Courses  is  to  create  a robust  digital  library  of  online  course  content,  providing  the  best  learning  experiences 
possible  for  any  institution,  instructor  or  student  who  wants  to  use  online  courses  to  enhance  learning  and 
teaching.  Our  short-term  goal  is  to  disseminate  the  NROC  guidelines  to  facilitate  collaboration  in  the 
course  development  community,  in  order  to  navigate  down  a pathway  of  continual  improvement  in  the 
quality,  interoperability  and  availability  of  complete  online  courses. 

Pedagogy  and  Content 

Integrated  Learning  Theory  and  Development  Guidelines 


Learning  Theory  Framework 

There  are  many  approaches  to  defining  learning  theories,  philosophies  and  best  practices.  The 
various  philosophies  of  learning  have  tended  to  create  silos  and  camps,  inspiring  debates  over  which  is 
most  effective  or  essential  to  learners’  achievement.  We  have  found  it  most  productive  for  our  purposes  to 
take  a “systems  approach”  to  learning  and  teaching  in  which  the  theories  and  philosophies,  with  their 
relevant  applications  and  data,  are  integrated  and  aligned  into  a unified  framework  of  learning.  NROC 
guidelines  have  therefore  been  designed  within  a comprehensive  framework  and  set  of  principles  that 
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assume  alignment  and  reinforcement  between  theories  of  learning  - as  elements  of  an  ecosystem  that 
require  reinforcing  relationships  between  elements  to  succeed  as  a whole. 

The  NROC  systems  approach  is  an  integration  of  learning  theories  focused  around  4 major 
themes:  learner  centered,  knowledge  centered,  assessment  centered  and  community-centered  approaches, 
(ref.  How  People  Learn , The  National  Research  Council).  We  have  derived  a list  of  some  of  the  significant 
attributes  under  each  approach  to  inform  our  guidelines.  We  are  asserting  that  the  best  online  learning 
experiences  involve  a mix  of  these  attributes,  and  that  they  should  reinforce  and  harmonize  with  one 
another. 


Learner  Centered 

o multiple  learning  styles 
o metacognition 
o feedback 
o interaction 
o relevance 

Knowledge  Centered 

o subject  expertise 
o concepts,  principles 
o subject  pedagogy 
o structure,  scaffolding 
o applied,  connected 

Assessment  Centered 

Community  Centered 

o multiple  types 

o 

social  context 

o understanding 

o 

cultural  context 

o concepts,  principles 

o 

real  world  connection 

o problem  solving 

o 

relevance 

o transfer 

o 

interaction 

Significant  Domains 

We  have  defined  ten  significant  domains,  or  dimensions,  within  which  principles  for  effective 
development  and  execution  of  online  courses  are  organized.  These  domains  were  chosen  by  synthesizing 
the  literature  on  learning  theories,  current  practices  and  experiences,  and  vetting  them  with  the  real  world 
issues  of  current  available  standard  technologies,  how  education  is  managed  and  delivered,  the  experiences 
of  teachers,  and  expectations  of  learners.  These  ten  domains  and  their  principles  include  the  following. 

Technology 

Description:  technology  environment  and  tools  proving  student  and  instructor  access  to  online 
course  content  and  experience. 

Principles:  Uses  standard  technologies,  bug  free,  runs  perfectly.  Interoperable  with  common 
learning  management  systems.  Provides  interaction  options  between  learners  content,  peers,  teacher, 
external  information  and  examples.  Clean  navigational  structure,  pathways,  organizational  hierarchy  and 
feedback  functionality. 

Rating:  (1-4)  4 Interoperable,  full  featured,  3:  Missing  major  feature,  2:  Limited  interoperability  and  limited 
features,  1:  Does  not  take  advantage  of  CMS  interaction  features/lacks  good  structure,  and  or  limited  in  features,  hard 
to  use. 

Content 

Description : Breadth,  depth,  approach  appropriate  for  course  level  and  subject  coverage. 

Principles’.  Clear  presentation  of  major  principles,  concepts,  framework  and  thinking  processes  of 
subject.  Incorporates  real  world  examples,  applications,  additional  sources  and  a variety  of  information 
types.  Learning  objectives  reflect  building  knowledge,  critical  thinking,  and  transfer.  Content  organized  in 
units  with  topics  and  subtopics.  Transparent  content  framework  that  shows  learning  chunks,  subject 
structure  and  associated  activities.  Coverage  maps  to  depth  and  breadth  of  generally  accepted  standards  for 
target  course  (such  as  established  by  AP  guidelines  & textbook  contents).  Includes  current,  socially  and 
culturally  relevant  context  and  connections.  Writing  tone,  level  and  style  clear,  informal  and  accessible  to 
learners. 

Rating:  (1-4)  4:  Complete  course  content  online,  3:  Majority  of  course  content,  linked  supplemental  content, 
2:  Areas  of  weak  coverage,  1:  Textbook  dependent  more  than  1/2  content. 
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Media 


Description:  use  of  multiple  media  types  to  present  content,  provide  activities  and  interactions. 

Principles:  Multi-modal  media  used  to  help  present,  and  work  with,  key  concepts,  examples, 
applications.  Provide  for  varied  learning  style  and  skill  approaches:  text,  graphics,  photos,  animation, 
simulation,  video,  audio  as  appropriate  for  conveying  subject.  Engaging,  motivating.  Coherent  arrangement 
of  media  types  designed  to  link  to  and  reinforce  each  other.  Learner  navigation  and  controls  available  and 
consistent. 

Rating:  (1-4)  4:  Good  design  and  pedagogical  use  of  4+  media  types,  3:  Good  pedagogical  use  of  3 or  fewer 
media  types,  2:  Weak  use  of  media,  1:  Limited  media,  and/or  poor  use. 

Design 

Description:  Overall  course  construction,  content  presentation,  navigation,  use  of  visual  elements, 
technical  features. 

Principles : Course  navigation  is  intuitive,  provides  logical  and  varied  paths  through  material. 
Helps  to  reveal  context  and  connections  of  topics.  Well-used  mix  of  media  for  content  presentation  to 
scaffold  new  understanding,  different  learning  modes  and  perspectives.  Elements  proportional, 
harmonious,  uncluttered  and  restrained.  Control  of  navigation  and  media  consistent,  clear  and  available  to 
learner. 

Rating:  (1-4)  4:  Well  executed  instructional,  information,  and  media  design,  3:  Design  weakness  can  be 
augmented  by  customization,  2.  Contains  design  flaws/does  not  enhance  experience,  1 : Design  hampers  learner 
experience. 

Pedagogy 

Description:  Application  of  universal  and  subject  specific  learning  theories  that  enable  effective 
learning  and  teacher. 

Principles:  Integration  and  alignment  of  1)  learner,  2)  knowledge,  3)  assessment  and  4) 
community  centered  pedagogical  approaches.  Incorporation  of  subject  specific  pedagogy.  Encourage 
learner  metacognition  - reflection  on  learning  process  and  progress.  Learner  as  driver,  teacher  as  facilitator, 
guide,  mentor.  Multiple  interaction  types,  safe  and  encouraging  environment.  Experience  designed  for 
learning  subject  framework  and  thinking  processes. 

Rating:  (1-4)  4:  Multiple  approaches  integrated  and  aligned  to  provide  rich  learning  and  teaching 
environment,  3:  At  least  three  well  executed  pedagogical  approaches,  2:  Lacking  learner  and  community  centered 
approaches,  1:  Lacks  coherent  pedagogical  approach. 

Interaction 

Description : Interaction  types  to  include  learners  with  content,  other  learners,  teacher  and  outside 
experts/information  through  use  of  technology,  course  design  and  activities. 

Principles:  Interactions  integrated  throughout,  providing  multiple  opportunities  to  construct 
meaning,  reflect,  examine  assumptions,  critique,  question,  and  transfer  knowledge,  initially  using  own 
language,  scaffolding  to  use  of  subject  language  and  thought  processes  while  advancing.  Learning 
community  of  social  interactions  between  peers,  with  teacher,  (most  motivational  learning  environment, 
best  opportunities  to  construct  understanding).  Students  and  teachers  create  activities  using  communication 
tools.  Performance  expectations,  and  instructor  availability  explicit. 

Rating:  (1-4)4:  Learning  activities  provide  multiple  opportunities  for  reflection  and  critical 
thinking  expressed  in  interactions  with  content,  peers,  instructor  and  external  examples,  flexible,  adaptable 
by  instructor  3:  Some  strong  use  of  learner  centered  interactions  with  peers  and  instructor,  2:  Weak 
opportunities  for  learner  centered  interactions,  1:  No  use  of  social  interaction  in  learning  activities  or 
assessments. 

Assignments 

Description:  A variety  of  activities  provided  for  students  to  interact  with,  use,  apply,  reflect  upon, 
and  build  content. 

Principles : Active  learning  opportunities  with  varied  assignments  consistent  with  content 
presentation,  learning  objectives,  and  assessment.  Provide  motivation,  encouragement  and  empowerment  to 
work  with  content.  Relevant  and  accurate.  Clear  instructions.  Appropriate  to  course  level  and  subject. 
Provide  opportunities  for  practice,  building  conceptual  frameworks,  knowledge,  critical  thinking, 
reflection,  understanding,  transfer  and  exploring  alternative  approaches  and  perspectives.  Different 
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learning  styles  and  skills  required,  using  mixed  media  and  communication  tools.  Expectations  transparent 
to  learners.  Collaborative  assignments  available  for  peer  teaching  and  assessment.  Adaptable  by  learner 
and  instructor. 

Rating:  (1-4)  4:  Rich  and  varied  use  of  individual  and  group  activities  to  build  subject  framework,  enhance 
practice  and  critical  thinking,  transfer,  expression  and  reflection  on  learning  process,  3:  Individual  and  group  modes  for 
practice,  application  and  critical  thinking,  2:  Requires  enhancement  of  activities  for  learner  centered  experience,  1: 
Lacks  assignments  requiring  reflecting  on  learning  process,  critical  thinking  and  learner  expression  of  content. 

Assessment 

Description:  A variety  of  assessment  types  providing  sufficient  testing  and  feedback  throughout 
the  course  to  show  knowledge  acquisition,  application  and  transfer. 

Principles:  Assessment  strategies  varied  and  consistent  with  content  presentation,  learning 
objectives  and  activities.  Formative  and  summative.  Relevant  and  accurate.  Clear  instructions.  Feedback 
from  quizzes  and  problems  provided  to  learner  throughout  course  to  enable  self-monitoring  of  progress. 
Appropriate  to  course  level  and  subject.  Show  critical  thinking,  reflection,  connections,  understanding  and 
transfer.  Expectations  and  grading  rubric  transparent  to  learners. 

Rating:  (1-4)  4:  Varied,  well  spaced  assessment  strategies,  multiple  learning,  styles,  immediate 
feedback  on  learning  process  and  progress,  reflects  mastery  of  learning  objectives,  3:  Varied  strategies, 
sufficient  feedback,  mapped  to  learning  objectives,  2:  Limited  assessment  options,  need  supplementing  to 
be  effective,  1:  Lacks  sufficient  assessment  to  measure  progress  towards  learning  objectives. 

Access 

Description:  Ability  of  students  with  some  learning  challenges  to  access  content,  including 
through  alternative  presentation  methods  and  support. 

Principles’.  Acknowledgement  of  need.  Effort  made  for  access  to  special  needs  learners. 
Alternatives  to  auditory  and  visual  content  are  available.  Consideration  for  navigation  by  motor  challenged. 
Color  conveying  meaning  is  available  without  color. 

Rating:  (1-4)  4:  Rich  content  and  learning  experience  supplemented  by  alternative,  parallel 
content  mode  accessible  to  auditory,  visually,  or  motor  challenged,  3:  Content  presentation  accessible  to 
visually  or  auditory  or  motor  challenged  2:  Some  content  not  available,  but  can  be  supplemented,  1:  Of 
limited  value  to  learning  challenged. 

Support 

Description:  Information  provided  to  learners  and  teachers  to  most  effectively  use,  facilitate, 
support,  and  design  the  online  course  experience. 

Principles:  Student.  Orientation  to  course  and  online  learning,  introduction  to  course  structure, 
guide  to  “experiencing  the  subject,”  complete  syllabus,  participation  expectations,  assignments,  grading 
policy,  skills  required,  prerequisites.  How  to  succeed  online  and  work  with  peers,  links  to  outside 
resources.  Teacher.  Guidelines  for  how  to  teach  online,  create  and  guide  collaborative  learning  activities, 
group  projects,  peer-to-peer  interactions.  How  to  create  a safe  and  encouraging  experience,  facilitate  and 
guide  students  to  construct,  control  and  share  their  learning  experiences.  Complete  instructors’  manual  with 
assignment  and  assessment  suggestions.  How  to  assess  discussions  and  group  projects.  Link  to  tips  and 
examples. 

Rating:  (1-4)  4:  Complete  set  of  learner  and  teacher  support  guidelines,  orientation,  examples  and 
external  resources  for  successful  online  learning  and  teaching  experience,  3:  Essential  learner  and  teacher 
support  materials:  orientation  and  guidelines  for  interaction,  2:  Support  material  needs  supplementing,  1: 
insufficient  support  material  for  learning  or  teaching  orientation  and  guidance  on  how  to  succeed  in  an 
online  course  environment. 


Design  and  Technology 

Interoperability 

Our  technical  guidelines  and  execution  have  been  designed  to  provide  as  much  practical 
interoperability  as  is  reasonably  possible  in  the  current  digital  environment.  We  (re)develop  courses  and 
learning  objects  to  fit  into  a flexible  schema  that  keeps  file  hierarchies,  content,  activities  and  assessments 
organized,  to  be  most  easily  ported  into  various  environments.  NROC  courses  operate  within  most 
commonly  used  CMS  tools  and  therefore  require  no  new  training.  With  a bit  more  effort  and  technical 
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assistance,  NROC  course  files  can  be  loaded  into  proprietary  and  open  source  learning  systems,  or  may  be 
simply  served  as  content  without  a CMS  in  an  open,  web-based  environment.  To  support  a wide  variety  of 
delivery  options,  NROC  provides  resources,  training  and  consulting  services  to  aid  course  library 
implementation. 


Interoperable  Delivery  Options 
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Customization 

To  address  some  of  the  cultural  and  technical  barriers  to  instructors  and  institutions  adopting 
courses  from  other  organizations,  we  have  designed  our  courses  and  processes  to  make  it  as  easy  as 
possible  for  customization.  Our  content  and  technical  guidelines  promote  a uniform  presentation  and 
organization  of  course  learning  objects  so  that  it  is  transparent  for  instructors,  or  development  partners  to 
customize  the  courses  within  learning  management  systems  by  turning  off  or  hiding  any  files  not  wanted, 
and  augmenting  the  content  presentations  and  activities  by  importing  new  files  into  the  course.  This  schema 
also  creates  independent  learning  objects  that  can  be  taken  out  of  the  course  sequence  to  build  new 
modules  or  lessons  around  or  enhance  a course  that  has  already  been  developed.  We  are  also  researching 
the  cultural  issues  in  the  way  of  more  substantial  course  exchanges  between  instructors  and  institutions.  We 
will  be  providing  training  seminars  and  an  online  learning  community  to  assist  in  learning  about  using  third 
party  online  content,  strategies  for  evaluating  materials,  and  practical,  hands-on  techniques  for  integrating 
all  of  these  resources  to  make  courses  more  engaging  and  effective. 


NROC  Technical  Development  Guidelines 

The  purpose  of  these  guidelines  is  to  provide  course  developers  with  an  understanding  of  the 
values,  characteristics  and  organization  of  the  course  content  in  the  National  Repository  of  Online  Courses. 
Course  developers  can  use  this  document  to  create  courses  that  are  consistent  with  the  course  architecture, 
distribution  characteristics  and  end-user  requirements  of  the  NROC  library. 

Background 

Our  technical  guidelines  are  meant  for  anyone  interested  in  developing  content  for,  or  contributing 
content  to,  the  NROC  course  library.  The  guidelines  are  not  a strict  set  of  technical  specifications,  but 
rather  a set  of  characteristics  and  values  that  all  courses  in  the  NROC  library  should  exhibit.  As  a set  of 
guidelines  that  represents  characteristics  and  values,  the  degree  to  which  any  course  adheres  to  the 
guidelines  can  vary.  Our  goal  in  publishing  the  guidelines  is  to  increase  the  degree  of  adherence  for  any 
NROC  course  candidate. 

NROC  Course  Characteristics 

The  NROC  course  library  has  broad  national  and  international  distribution.  Distribution  typically 
includes  sending  courses  on  CDs  or  DVDs  to  be  installed  and  hosted  on  a licensee’s  server.  Candidate 
NROC  courses  should  be  free  of  any  licensing  or  technical  restrictions  that  would  prohibit  this  type  of 
distribution. 
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Most  NROC  courses  contain  a great  deal  of  multimedia  content.  Multimedia  content  can  be 
instructional^  effective,  but  it  is  very  difficult  to  edit-  it  requires  special  software,  skills  or  training,  and  is 
some  cases,  access  to  source  code  or  source  files.  NROC  courses  break  down  the  multimedia  instructional 
content  into  discrete  lessons  (learning  objects)  that  can  be  deleted,  reorganized,  or  repurposed,  giving  an 
institution  or  instructor  the  greatest  flexibility  and  degree  of  modification  available  without  directly  editing 
the  content. 

Multimedia  content  also  requires  a great  deal  of  storage  space  on  a web  server.  Because  most 
institutions  offer  multiple  sections  or  sessions  of  a single  class,  each  requiring  their  own  copy  of  a course 
within  a CMS,  the  NROC  course  hosting  model  places  all  multimedia  course  assets  on  a single  central 
content  server.  This  allows  the  same  multimedia  learning  objects  to  be  referenced  by  each  copy  of  a course 
without  duplicating  the  content  in  each  course.  Since  the  multimedia  learning  objects  are  not  directly  edited 
by  the  instructor  of  each  course  session,  only  one  copy  of  each  learning  object  is  required  on  the  central 
content  server. 

NROC  courses  are  designed  for  use  with  a Course  Management  System.  Most  NROC  courses 
have  been  pre-loaded  on  WebCT,  Blackboard  and  eCollege  to  facilitate  installation  on  these  Course 
Management  Systems.  All  course  content  is  provided  in  one  or  more  formats  that  are  compatible  with  most 
other  Course  Management  Systems.  In  addition  to  interoperability,  NROC  courses  seek  to  leverage  the 
basic  functionality  provided  by  CMSs.  NROC  courses  use  the  CMS  system  to  provide  the  course 
organization,  display  the  course  menu,  manage  communication,  manage  collaboration  activities,  and 
deliver  assessments.  By  relying  on  the  CMS  for  this  functionality  and  providing  instructional  and 
assessment  content  that  is  in  a format  that  can  be  used  by  a CMS,  NROC  courses  inherit  the  features  of  the 
CMS  system.  Instructors  can  use  the  interfaces  provided  by  their  CMS  to  edit  the  course  structure,  add 
content,  change  the  scoring  and  weighting  of  assignments  or  edit  assessment  items. 

The  NROC  course  model  relies  on,  and  seeks  to  support,  the  role  of  the  instructor  in  a course.  In 
addition  to  the  course  modification  support  and  flexibility  described  already,  NROC  courses  include  a 
Teacher’s  Guide,  answer  keys  and  grading  rubrics.  The  Teacher’s  Guide  for  each  course  may  include  notes 
and  teaching  tips  for  each  section  of  the  course,  alternate  and  supplemental  assignments  or  assessments  and 
correlations  to  multiple  textbooks. 

The  following  section  provides  more  details  on  NROC  course  requirements,  characteristics  and 
guiding  principles. 

Server  Requirements 

NROC  courses  are  typically  hosted  by  licensees  of  the  NROC  library.  NROC  courses  are 
delivered  using  a standard  HTTP  web  server  like  Internet  Information  Server  with  no  special  port  or 
firewall  requirements.  Courses  should  not  rely  on  any  proprietary  server-side  technologies  or  database 
connections.  Note  that  NROC  courses  are  designed  for  use  with  a Course  Management  System  that  will 
have  its  own  list  of  server  requirements.  See  Appendix  B:  System  Requirements  for  more  information  on 
this  topic. 

Client  Requirements 

NROC  courses  use  standard  web  technologies  including  HTML,  FLASH,  Java,  JavaScript  and 
PDF.  They  are  designed  to  run  in  standard  browsers  including  Internet  Explorer,  Safari,  Netscape  and 
Firefox  on  Windows  and  Macintosh  operating  systems.  Courses  should  avoid  reliance  on  non-standard 
browser  plug-ins  and  extensions.  In  order  to  minimize  the  NROC  client-side  requirements,  NROC 
recommends  the  use  of  QuickTime  for  delivery  of  web-based  video  content 

Licenses  and  Royalties 

Candidate  courses  should  not  require  any  software  or  media  licenses  or  royalties  associated  with 
NROC  distribution  of  the  course.  NROC  courses  should  not  rely  on  any  proprietary  software  or  technology 
that  requires  a software  license  and  royalty  payment.  Course  media  elements  including  images,  video, 
audio  and  animation  should  not  require  any  re-licensing  or  special  permissions  by  NROC  library 
subscribers. 

Complete  Courses 

NROC  courses  should  be  complete  in  their  scope  and  coverage  of  the  topic  presented.  While  the 
course  may  require  the  use  of  a textbook,  labs  or  other  supplemental  materials,  these  materials  should  not 
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be  the  primary  instructional  resource  for  any  section  of  the  course.  NROC  courses  are  designed  to  support  a 
distance  learning  education  model  where  the  course  can  be  completed  on-line,  with  moderation  by  an 
instructor.  NROC  courses  can  also  be  used  as  a collection  of  resources  to  support  other  use  cases,  including 
support  of  tradition  classroom  courses  and  teacher  education  programs. 

Reuse 

NROC  courses  employ  an  object-based  architecture.  The  courses  are  built  as  a collection  of 
lessons,  assignments,  labs  and  assessments  that  are  organized  within  units  and  chapters.  Each  course 
element  should  be  self-contained  - it  should  not  rely  on  the  presence  of  other  discreet  elements  to  be 
technically  or  instructionally  functional.  Instructional  objects,  especially  multimedia  content  that  is  not 
readily  editable  by  an  instructor,  should  be  small  enough  to  be  viewed  in  one  instructional  period  (about  45 
minutes).  However,  to  increase  the  potential  for  reuse,  while  maintaining  instructional  context  and 
minimizing  the  complexity  of  the  course  map,  learning  objects  will  ideally  represent  6 to  12  minutes  of 
instruction.  The  object-based  NROC  course  design  allows  course  content  to  support  a number  of 
educational  uses.  Packaging  a course  as  a collection  of  discreet  elements  allows  an  instructor  to  modify  and 
augment  a course  by  deleting,  editing,  reorganizing  and  adding  content. 

CMS  Friendly 

NROC  courses  consist  of  a collection  of  instructional  and  assessment  elements  that  are 
represented  in  formats  that  are  compatible  with  most  other  Course  Management  Systems.  NROC  courses 
use  the  CMS  system  to  provide  the  course  organization  and  display  the  course  menu.  NROC  courses  are 
designed  to  leverage  the  functionality  provided  by  the  CMS  system.  Course  content  including  discussion 
questions  and  collaboration  projects  are  designed  to  use  the  communication  tools  provided  by  most  CMS 
systems. 

Assessment  Delivery 

NROC  course  assessments  are  delivered  with  the  assessment  tools  provided  by  the  CMS. 
Assessment  item  types  are  limited  to  those  commonly  supported  by  CMS  assessment  tools  and  include 
multiple  choice,  short  answer,  and  matching.  The  assessment  content  is  provided  in  a number  of  formats  to 
facilitate  importing  the  content  into  the  CMS.  The  available  formats  include  RTF,  XML  (IMS  QTI)  and 
Respondus  (a  commercial  assessment  management  and  development  tool  - www.respondus.com).  Answer 
keys  and  grading  rubrics  are  provided  for  all  course  assessments. 

Role  of  the  Instructor 

While  NROC  courses  represent  an  organized,  complete  and  in-depth  treatment  of  the  topic,  the 
course  instructor  is  the  final  arbiter  of  what  is  presented  to  the  student.  NROC  courses  can  be  used  as  is, 
edited,  augmented,  combined  with  an  existing  course  or  used  as  a learning  object  repository.  To  support  the 
instructor,  NROC  courses  include  a Teacher’s  Guide.  The  Teacher’s  Guide  should  contain  notes  and 
teaching  tips  for  each  section  of  the  course,  a pacing  guide,  rubrics  for  grading  assignments  and 
assessments,  alternate  and  supplemental  assignments  or  assessments  and  correlations  to  multiple  textbooks. 

NROC  Course  Structure 

Home 

The  course  Home  serves  as  an  introduction  to  the  course.  The  Home  section  includes  the  course 
title,  a brief  description  of  the  course  content  and  course-level  objective  and  outcomes. 

Syllabus 

Each  NROC  course  has  a Syllabus  section  that  contains  information  about  the  course  elements, 
course  organization,  and  textbooks  that  may  be  used  with  the  course.  All  of  the  textbooks  listed  in  this 
section  have  been  “mapped”  to  the  course  content.  The  syllabus  section  may  also  contain  other  resources 
such  as  web  links  or  class  discussion  topics. 

Chapters 

NROC  course  content  is  organized  by  Chapter  (in  some  NROC  courses  a collection  of  Chapters 
may  be  organized  into  a Unit).  A typical  semester  course  will  have  between  10  and  15  Chapters.  Each 
Chapter  contains  the  Readings,  Lessons,  Assignments  and  Assessments  that  make  up  the  course  content. 
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Additionally,  each  will  typically  contain  a midterm  and  final  exam.  The  diagram  below  shows  the  structure 
of  a typical  NROC  course. 
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Abstract 

A grid  is  a new  paradigm  of  distributed  computing  to  share  and  collaborate  with  decentralized 
resources.  A Web-Service-based  OGSA  approach  for  a grid  can  improve  the  extensibility  and 
interoperability  of  a grid  system.  This  paper  emphasizes  technical  features,  the  architecture,  and 
advantages  of  OGSA  compared  to  distributed  networks.  It  also  analyzes  educational  resources  of  South 
China  Normal  University  and  describes  a solution  to  resources  integration  according  to  the  IBM  product 
OGSA-based  Service  Domain. 


Introduction 

With  the  development  of  technology,  Grid  Computing  has  exceeded  its  original  academic  field 
and  has  greatly  influenced  other  fields.  In  the  field  of  Grid  Computing,  many  forums,  such  as  Global  Grid 
Forum,  have  been  founded,  and  lots  of  companies  such  as  IBM,  Sun,  etc.,  are  doing  research  on  it. 
Moreover,  new  research  institutes  and  industrial  organizations  have  been  investigating  Grid  Computing. 
Research  on  Grid  Computing  has  become  a hotspot  nowadays. 

Campus  Grid  is  one  of  the  main  applications  of  Grid  Computing  to  scientific  computation.  The 
university  is  the  chief  scientific  and  academic  institute  in  our  country,  which  has  been  greatly  invested  by 
the  government.  Grid  Computing  becomes  an  ideal  technology  for  the  campuses  with  rich  computational 
resources  and  scientific  research  projects  calling  for  large-scale  computation.  So  far,  researches  have 
focused  on  some  powerful  research  institutes  of  high  performance  computing,  such  as  Tsinghua  University, 
Peking  University,  etc.  As  one  of  the  "Project  211"  universities,  South  China  Normal  University  is  already 
on  the  initial  stages  of  applying  grid  technology.  The  OGSA  (Open  Grid  Services  Architecture)  Grid 
computing,  based  on  the  two  mainstream  technologies:  Grid  Computing  and  Web  Service,  has  great  effect 
on  integrating  existing  educational  resources  properly. 

The  Architecture  and  Features  of  OGSA 

Technical  theories  of  Grid  Computing  have  not  established  formal  standard.  However,  in  the  view 
of  core  technology,  Globus  Toolkit  (GT)  has  been  the  virtual  standard  of  Grid  Computing.  GT1  was  built 
up  in  1999,  and  the  following  GT2  is  a paragraph  of  service  set  and  software  library  for  the  grid  and  Grid 
Computing,  which  is  based  on  community,  open  framework  and  open  codes.  In  2002,  IBM  and  Globus 
announced  OGSA,  which  drew  the  outline  of  GT3.0.  OGSA  has  been  implemented  by  GT3,  some  other 
open  codes  and  commercial  system.  So  far  GT4  has  been  released  to  make  a further  improvement  of 
OGSA. 

The  Main  Features  and  Key  Technologies  of  Grid  Computing 

The  main  features  [1]  of  Grid  Computing  include:  heterogeneity,  scalability,  adaptability, 
unpredictability  of  structure  and  multi-level  management  domain,  etc.  The  key  technologies  [2]  include:  (1) 
Nodes  of  a grid,  the  resource  providers,  including  highend  servers,  clusters  and  so  on.  (2)  Broad-band 
network,  providing  necessary  means  of  high  performance  communication  in  Grid  Computing.  (3)  Tools  of 
resource  management,  task  management  and  task  dispatching,  which  actively  dispatch  tasks  according  to 
the  current  load  status  of  the  system  to  improve  function  efficiency.  (4)  Inspecting  tools,  helping  users  take 
full  advantage  of  resources.  (5)  Visualization  tools  of  Application  Layer,  providing  intuitionistic  visualized 
information  and  friendly  user  interfaces. 
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OGSA 


OGSA  is  the  latest  and  the  most  important  grid  architecture,  which  is  put  forward  after  the  ‘five- 
layered Sandglass  Structure’.  It  is  called  the  Next  Generation  Grid  Architecture. 

So  far,  the  three  main  components  of  OGSA  are  Open  Grid  Services  Infrastructure  (OGSI),  OGSA 
Services  and  OGSA  Modules.  OGSA  is  based  on  Web  Service,  which  is  a standard-based  and  widely- 
disposed  distributed  computing  pattern,  providing  a basic  mechanism  to  describe  and  call  on  grid  service. 
With  the  core  interface  set  of  OGSI,  semantic  meaning  of  Web  service  and  other  important  services  could 
be  standardized,  so  that  the  service  can  be  virtualized  and  manipulated  with  another.  The  Web  Services  in 
accord  with  the  standard  of  OGSI  are  called  Grid  Service  [1].  In  OGSA,  everything  is  considered  as  Grid 
Service.  Grid  Service  can  be  congregated  by  different  ways  to  meet  the  needs  of  the  virtual  organization, 
and  can  be  defined  by  the  virtual  organization  partly  according  to  its  manipulation  and  shared  services. 

OGSA  is  composed  of  four  main  layers,  from  the  bottom  to  the  top  as  follows:  resources  (physical 
resources  and  logical  resources),  Web  Service  and  OGSI  extensions  which  define  the  grid  service,  OGSA 
architected  services,  and  grid  applications. 

Two  backbone  technologies  of  OGSA  are  Grid  Technology  (Globus 
Toolkits)  and  Web  Service.  Globus  is  the  solution  to  gird  technology  which  has 
been  widely  accepted  by  the  fields  of  science  and  engineering;  Web  Service  is  a 
standard  application  framework  for  network  storage.  Globus  is  the  collection  of 
services  based  on  community,  open  architecture  and  open  codes,  and  it  supports 
software  library  for  network  applications  and  grid  applications,  which  solve  the 
problems  such  as  security,  information  detection,  resource  management,  data 
management,  communications,  error  detection  and  the  transplantability,  etc.  Web 
Service  is  used  to  describe  the  interface  of  software  components,  the  methods  of 
accessing  components  through  interactive  protocols,  and  detective  method  of 
identifying  the  related  service  providers. 


Applications 


OGSA  Architected  Services 


OGSl-Opcn  Grid  Services  Infrastructures 


Web  Services 


Storage 


] 


Physical  Resources  and  Logical  Resources 


Fig.  1 Open  Grid  Services  Architecture 


The  Comparison  between  OGSA  and  Sandglass  Structure  [3] 

Two  chief  standards  appear  in  the  development  of  Grid-OGSA  and  Sandglass  Structures,  and  they 
have  their  own  features  and  advantages.  OGSA  develops  on  the  basis  of  Sandglass  Structure.  Moreover, 
OGSA  has  its  distinctive  advantages  of  complicated  distributed  resources.  The 
comparison  between  them  shows  as  following,  Table  1. 


The  Comparison  between  OGSA  and  Web  Service 

OGSA  combines  two  mainstream  technologies:  Grid  Computing  and 
Web  Service.  It  utilizes  Grid  Computing  to  extend  the  range  of  Web  Service,  and 
it  carries  out  the  mapping  and  the  application  from  the  service  functions  to  the 
original  platform,  therefore,  it  makes  the  integration  of  the  services  and  the 
resources  perfectly.  Although,  OGSA  is  quite  similar  to  Web  Service  not  only  on 
the  technical  concept,  but  also  on  the  concrete  implement.  However,  there  are 
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3.  The  Solution  to  Education  Resources  Integration 
The  Introduction  to  Application  and  Development  of  Grid  Computing  in  Education 

Grid  Computing  has  been  used  in  the  fields  of  education  in  China.  The  ChinaGrid  project,  carried 
out  by  the  ministry  of  education  in  (the  tenth  five-year  project)  "Project  211",  is  one  of  the  largest  grid 
projects  in  the  world.  ChinaGrid  is  implemented  by  twelve  universities,  including  Huazhong  University  of 
Science  and  Technology,  Tsinghua  University,  Peking  University,  etc.  It  can  obtain  resources  that  have  not 
been  used  completely,  from  different  computer  systems,  and  provide  them  to  users  when  necessary  so  as  to 
form  a unified  virtual  system.  When  the  grid  network  is  finished,  the  resources  of  one  hundred  "Project 
211"  universities  will  be  shared  widely  over  the  whole  country.  As  one  of  the  "Project  211"  universities, 
South  China  Normal  University  does  some  researches  and  carries  out  some  experiments  and  applications 
about  grid  computing  according  to  practical  situations. 

South  China  Normal  University  has  one  local  chief  campus  and  two  other  districts  (U-City  and 
Nanhai  College).  The  educational  resources  of  the  chief  campus  contain  four  parts:  Network  Center,  IT 
Center,  School  of  Distance  Learning  and  the  Library.  Network  Center  is  the  access  point  to  manage  IT 
Center,  the  Library  and  School  of  distance  learning.  IT  Center  manages  the  teaching  building,  student 
dormitories  and  staff  dormitories;  School  of  Distance  Learning  manages  the  local  institute  and  the 
education  of  distance.  Each  of  these  four  main  parts  has  several  personal  computers;  both  IT  Center  and 
School  of  Distance  Learning  have  some  equipment  such  as  minicomputers,  servers,  etc. 

The  Analysis  of  Resources  Integration 

When  outer  users  access  the  educational  resources  from  SCNU,  they  must  get  through  a Network 
Portal.  Identified  by  the  firewall,  the  request  from  the  users  can  be  attached  to  the  servers  of  Network 
Center  or  School  of  Distance  Learning.  The  Web  Server  communicates  with  the  Service  Domain  server, 
Virtualization  Layer  of  Service  Domain  server  transfers  the  requests  of  users  into  service  requests  that  can 
be  recognized  by  Service  Layer  via  Web  application  programs.  Then  the  Hubs  of  each  level  of  Service 
Domain  distribute  the  service  requests  to  the  grid  nodes.  Finally,  the  computational  result  will  return  to  the 
users. 

Generally  speaking,  the  campus  educational  resources  are  idle  most  of  the  time  and  the  rate  of 
utilization  is  not  high.  When  a great  number  of  scientific  computing  must  be  done,  we  can  use  grid 
technology  based  on  OGSA  and  its  corresponding  products  to  allocate  the  resources  and  to  communicate. 
But  some  security  problems  occur.  Hence,  one  of  the  components  of  OGSA:  OGSI  and  its  corresponding 
protocols  are  put  forward  to  solve  these  problems. 


Service  Domain  [4] 

Each  part  of  the  campus  network  of  SCNU  has  its  distinctive  functions  and  responsibilities. 
Therefore,  they  provide  different  services  to  the  users.  Considering  the  situations,  we  choose  Service 
Domain  based  on  OGSA  Architecture  and  Service-oriented,  which  is  put  forward  by  IBM  Company  in 
order  to  integrate  the  resources. 


Architecture 

Service  Domain  [4]  is  a technology  for  Service  Grid.  It  has  an  access 
point  to  integrate  the  relevant  Web  Service,  and  it  adopts  the  theory  of 
independent  computing  to  combine  Web  Service  and  Service  Grid  together.  It 
consists  of  two  layers:  Service  Domain  of  Service  Layer  and  On  Demand  Service 

Grid  (ODSG)  of 
Virtualization  Layer  that 
varies  when  necessary. 
These  two  layers  constitute 
the  perfect  service-oriented 
architecture.  Service 
Domain  virtualizes  service 
integration  into  the  grid; 
whereas  ODSG  virtualizes 
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Fig.  2 SO  A of  Service  Domain 


Fig.  3 ODSG  Architecture 


the  grid.  These  functions  offer  a feasible  service  running  and  maintenance  mechanism.  Figure  2 shows 
Service  Oriented  Architecture  (SOA)  of  Service  Domain. 

Being  Virtualization  Layer  of  Service  Grid,  ODSG  integrates  Service  Domain  with  local  Hosting 
technology,  and  provides  the  users  with  heterogeneous  services  integrated  in  different  service  groups 
(OGSI  portType).  The  Hub  provides  its  deputizing  services  with  the  single  external  logical  interface.  The 
Main  Hub  can  also  transmit  the  requests  of  the  service  to  the  Sub  Hubs,  which  constitutes  Tree  Structure  of 
Service  Routing.  Figure  3 shows  ODSG  Architecture. 

ODSG  has  a feature  of  distributed  dynamic  service  publication,  thus  the  concrete  service  can  be 
released  on  its  affiliated  Sub  Hub  which  the  service  has  been  registered  into,  but  the  service  can  also  be 
called  by  Main  Hub.  What's  more,  ODSG  has  called  portals  of  public  grid  for  service  invocation  sent  by  a 
unified  interface,  moreover,  the  portal  is  managed  by  Main  Hub  to  monitor  the  status  of  system 
performance. 


Design  and  Implementation 

A Sub  Hub  can  be  set  up  in  each  part  of  the  university.  For  example, 

SCNU  could  build  up  three  Sub  Hubs  and  register  the  resources  of  the  university 
in  the  form  of  services  to  their  own  Sub  Hub,  respectively.  General  applications 
can  be  registered  into  Web  Service,  while  computing  resources  can  be  registered 
into  Grid  Service.  Inside  each  university,  resources  are  utilized  by  gaining  the 
services  of  the  Sub  Hubs.  The  services  which  have  the  same  functions  will  be 
registered  to  the  unified  interface  (portType),  and  then  the  Sub  Hub  will  choose 
the  best  service  provider  for  the  users  according  to  the  current  load  and  the 
customized  management  strategy. 

Each  Sub  Hub  can  also  be  registered  to  the  Main  Hub  so  that  the 
resources  of  each  Sub  Hub  can  be  recognized  and  employed  by  other  parts  of  the 
Sub  Hub.  For  example,  a user  wants  to  search  an  article,  and  IT  Centre  and 
School  of  Distance  Learning  provide  this  service.  Both  of  their  invocation 
interfaces  are  the  same,  but  the  quality  of  the  services,  such  as  the  response  time, 
is  quite  different.  When  the  users  log  in  Main  Hub  and  request  the  course  service, 

Main  Hub  will  choose  a best  Sub  Hub  for  them  according  to  the  privilege  of  them 
and  the  current  load  of  the  services.  So  the  users  can  ignore  the  services  from 
which  of  the  nodes. 

The  applications  of  Service  Domain  are  based  on  Web  Service,  so  users 
need  not  spend  lots  of  time  to  develop  the  application  platform,  but  they  can  concentrate  on  the 
development  of  the  business.  What’s  more,  scheduling  and  management  can  be  implemented  by  registering 
services,  configuring  the  strategy  files,  etc. 


Fig.  4 Educational  Resources  Integration 
with  Service  Domain  Technology 


Conclusion 

According  to  the  design  of  educational  resources  integration  of  South  China  Normal  University, 
Grid  technology  can  be  employed  to  integrate  and  optimize  the  educational  resources,  so  that  we  are  able  to 
achieve  the  goals  of  maximum  utilizing  the  resources  in  reason  and  scientific  calculating  in  a great  amount. 
Meanwhile,  we  come  up  with  some  ideas  about  the  campus  grid: 

(1) The  university  has  advantages  of  putting  Grid  Computing  in  practice,  and  it  is  in  need  of 
sharing  various  kinds  of  data,  resources  (such  as  sharing  data,  computational  resources,  storage  resources, 
and  application  resources),  and  complicated  computation.  What’s  more,  the  university  has  a great  technical 
power,  so  that  they  are  sensitive  to  the  new  technology. 

(2)  Applying  the  grid  to  campus  is  a crucial  measure  to  catch  up  with  the  development  of  the  next- 
tide  Internet  technology. 

(3)  Applying  the  grid  to  campus  is  a proper  method  to  upgrade  digitized  campus -information 
fundamental  structure. 

(4)  At  the  beginning,  we  can  build  up  several  Sub  Hubs  to  construct  the  grid,  and  then  enlarge  the 
size  gradually  to  optimize  the  education  resources. 

(5)  Wireless  Network  is  the  appropriate  way  for  access. 

With  the  development  of  the  wireless  technology,  wireless  network  will  become  an  indispensable  part  of 
network  communications  at  campus,  or  even  in  the  whole  society.  The  wireless  access  equipments  such  as 
Bluetooth,  notebook,  moveable  PC,  multimedia  information  terminal,  give  new  challenges  and  good 
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opportunities  to  applications  of  Grid  technology.  A series  of  technologies,  for  example,  the  Wireless  Grid, 
will  have  a bright  and  flourish  prospect  in  the  near  future. 
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Seamless  Technological  Curriculum  Pathways  the  Engage  Curricula 

Redesign 

Rayton  Sianjina 

As  new  technologies  continue  to  emerge  and  students  require  more  flexibility  in  course 
scheduling,  institutions  of  higher  education  are  striving  to  accommodate  these  students.  As  a result 
institutions  of  higher  education  have  responded  by  implementing  online  learning  system.  However,  for  an 
academic  program  or  course  to  successfully  make  the  transition  from  the  traditional  to  the  online  delivery 
methodology,  there  are  many  factors  to  be  considered  carefully.  Accordingly,  Levy  (2003)  faculty 
members  are  faced  with  a number  of  new  challenges  when  teaching  an  online  learning  course  as  opposed  to 
a traditional  one.  These  challenges  include:  “the  administration  or  management  of  online  courses;  the 
course  layout  and  design;  the  best  delivery  method  for  the  content,  such  as  text,  graphic,  audio,  or  video; 
the  various  communication  methods  that  the  students  will  use  such  as  email,  discussion  boards,  and  chats; 
ways  to  increase  and  maintain  student  involvement;  appropriate  student  assessments  for  online  learning; 
and,  a working  knowledge  of  all  the  technologies  being  implemented  in  the  online  course”  (p.  22). 

Additionally,  college  administrators  are  increasingly  putting  pressure  on  faculty  to  participate  in 
distance  learning  and  other  technology  related  endeavors.  Most  faculty,  however,  have  not  responded  as 
quickly  and  enthusiastically  as  administrators  would  like.  For  a variety  of  reasons,  faculty  members  resist 
efforts  to  engage  in  distance  learning.  Specifically,  faculty  members  have  expressed  the  concern  of  a lack 
of  institutional  support,  the  change  in  interpersonal  relations,  and  quality  in  program  offerings  via  distance 
education  technologies. 

In  an  effort  to  reach  non-traditional  students,  Delaware  State  University,  just  like  other  institutions 
of  higher  learning,  through  office  the  of  Distance  Education  and  Learning  Technologies,  is  proactively 
providing  its  faculty  with  effective  and  meaningful  professional  development  workshops  in  order  to  assist 
them  in  transitioning  from  “brick  wall  to  click  wall”  learning  and  teaching  community.  This  office  has 
developed  strategies  that  attract  faculty  to  participate  in  course  redesign  and  development  activities.  Faculty 
members  are  provided  with  incentives  in  order  to  engage  them  in  this  process.  In  the  next  sections,  of  this 
paper,  discussions  of  various  strategies  are  given  in  detail. 

Instructional  Support 

All  humans  operate  and  are  motivated  by  positive  reinforcement.  Faculty  members  must  be  given 
incentives  for  their  efforts  and  involvement  with  distance  learning  technologies.  Institutional  support  is  a 
critical  part  of  course  to  course  redesign  for  the  new  technology,  especially  distance  education. 

Certainly,  in  order  to  be  successful  as  an  online  instructor,  faculty  members  need  to  have  some 
understanding  of  pedagogy  as  it  relates  to  online  instruction.  These  instructors  need  to  know  how  to 
transition  from  traditional  lectures  to  more  interactive  lessons  that  encourage  students  to  be  active 
participants.  Furthermore,  instructors  must  do  more  than  develop  new  technical  skills.  Online  development 
and  delivery  does  require  new  redesigned  pedagogical  approaches.  Certainly,  at  best,  it  necessitates  for  an 
institution  to  rethink  its  overall  academic  mission.  Additionally,  college  administrators  are  increasingly 
putting  pressure  on  faculty  to  participate  in  online  learning  and  other  technology  related  endeavors.  Most 
faculty,  however,  have  not  responded  as  quickly  and  enthusiastically  as  administrators  would  like.  Faculty 
members  express  serious  concerns  and  reservations  regarding  the  effectiveness  of  online  learning.  For 
example,  the  1998-99  national  study  of  faculty  conducted  by  UCLA’s  Higher  Education  Research  Institute 
(HERI)  revealed  that  two-thirds  of  college  and  university  faculty  found  keeping  abreast  of  information 
technology  stressful,  rating  it  above  research/publishing  demands,  teaching  load,  and  the  tenure/promotion 
as  a significant  stressor  (HERI,  1999).  Not  only  that,  online  delivery  challenges  traditional  assessment 
practices  and  brings  together  teams  of  people,  each  with  unique  skills,  into  a course  redesign  and 
development  team.  Clearly,  in  order  for  online  instruction  delivery  to  be  effective,  it  is  inevitable  for 
course  redesign  to  take  place.  Certainly,  online  instruction  calls  for  rethinking  of  the  traditional  instruction 
practices. 

Furthermore,  in  this  Internet  era,  institutions  of  higher  learning  have  witnessed  the  exciting 
convergence  of  content,  technology,  and  effective  learning  and  teaching  strategies.  Today,  the  academe, 
faculty,  and  students  benefit  from  access  to  the  rich  contents  and  treasured  resources  from  various 
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technologies.  The  old  model  of  “owning”  a collection  or  site  has  given  way  to  “sharing,”  and  the  new 
emphases  have  shifted  from  a large  physical  classrooms  to  virtual  classrooms. 

Although  college  administrators  are  increasingly  putting  pressure  on  faculty  to  participate  in 
online  learning  and  other  technology  related  endeavors,  for  a variety  of  reasons,  faculty  resistance  to 
participate  in  this  arena  is  high.  While  individual  faculty  members  may  have  individual  reasons  to  resist 
participating  in  the  latest  wave  of  online  education,  Bower  points,  “there  are  several  reasons  why  faculty  in 
general  resist  distance  education. 

The  rapid  development  of  digital  communication  technologies  allows  institutions  of  higher 
education  to  move  beyond  the  “brick  and  mortar”  restrictions  of  place  and  time  to  serve  a larger,  broader, 
and  more  diverse  population.  Although  it  is  not  a new  idea,  the  implementation  of  distance  learning  in 
higher  education  has  increased  exponentially. 

Certainly,  the  most  common  type  of  training  needed  by  participants  is  the  use  of  course 
management  software  (CMS).  CMS  systems  provided  by  companies  such  as  Blackboard  and  WebCT 
integrate  responsive  instructional  functions.  Typically,  most  CMS  are  characterized  by  user-friendly 
interface  that  facilitate  access  to  online  lectures,  assignments,  moderated  discussions,  quizzes,  grades  and 
synchronous  and  asynchronous  chat  sessions. 

Pedagogical  Training 

Clearly,  techniques  that  are  effective  in  the  traditional  classroom  are  not  necessarily  effective  in  an 
online  environment  (White  and  Weight  2000).  Online  classroom  provides  the  opportunity  to  establish  a 
unique  community  of  learners,  but  such  a technique  requires  facilitating  a development  of  a community 
very  different  from  those  that  work  in  the  traditional  face-to-face  classroom  setting  (Palloff  and  Pratt  2000). 
Faculty  members  who  are  experienced  and  successful  in  the  traditional  classroom  cannot  intuitively  and 
automatically  make  the  transition  to  the  online  environment  (Harasim  1990).  They  need  professional 
development  in  order  to  learn  what  works  and  what  doesn’t  work  online.  Therefore,  faculty  members  who 
do  not  want  to  “reinvent  the  wheel”;  pedagogically  will  sooner  or  later  learn  the  online  environment 
techniques,  in  order  for  their  students  to  succeed.  Additionally,  faculty  members  recognize  that  formal 
training  can  save  them  time  in  the  long  run,  as  well  as  allow  them  to  become  more  effective  and  successful 
online  teachers  faster. 


Recommendations 

Training  for  faculty  members  to  teach  online  should,  at  the  minimum,  contain  four  major 
components.  Namely:  Technical  training,  Pedagogical  training,  online  course  work,  and  Incentives. 

Technical  training  should  include  both  the  Course  Management  System  (CMS)  that  will  be  used  to 
deliver  the  online  course  and  the  use  of  other  software  that  facilitates  communicating  through  the  Internet, 
such  as  Blackboard  or  WebCT.  In  the  case  of  Delaware  State  University,  Blackboard  is  the  course 
management  system  that  the  university  uses.  Other  types  of  software  can  also  used,  for  example,  interactive 
screen  captioning  software  is  invaluable  in  provided  training.  The  captioned  screens  seem  interactive  even 
when  one  is  working  off-line. 

A national  Center  for  Education  Statistics  (NCES)  1997  report  indicated  that  about  60  percent  of 
higher  education  institutions  provide  training  opportunities  for  distance  learning  faculty  in  this  area.  This 
means  that  40  percent  of  the  institutions  offering  distance-learning  courses  asked  faculty  to  teach  these 
courses  without  providing  any  special  preparation  for  the  experience.  In  addition  to  expertise  in  their 
content  areas,  faculty  members  need  to  attain  a level  of  proficiency  with  the  computer  technologies  needed 
to  develop  and  deliver  online  instruction.  Online  educators  need  to  be  competent  in  using  technology  as  a 
means  for  effective  instruction  (Floyd,  2003).  Staff  development  is  essential  for  the  successful  movement 
from  the  traditional  classroom  to  a distance-learning  environment. 

Online  Course  Work 

One  incentive  used  by  institutions  to  encourage  the  faculty  to  get  involved  in  institutional 
initiatives  is  workload  adjustment.  However  the  NEA  study  (2000)  indicated  that  course  reduction  was  not 
provided  to  the  large  majority  (84  percent)  of  the  faculty  in  its  national  survey.  Perhaps  this  is  because  one 
of  the  reasons  for  expanding  distance  learning  and  use  of  educational  technology  is  its  increased 
productivity.  If  institutions  provide  release  time  to  faculty  preparing  distance  learning  courses,  they  may 
need  to  demonstrate  increased  productivity  through  other  means  such  as  increased  student-faculty  ratio  in 
distance  learning  classes. 
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Compensation  for  online  instruction  is  an  area  of  concern  for  faculty.  According  to  a survey 
conducted  by  the  National  Education  Association  (2000),  63%  of  distance  learning  faculty  is  compensated 
for  a distance-learning  course  as  if  it  were  a normal  course  even  though  online  instruction  takes  more 
preparation  time.  Sellani  and  Harring  (2002)  note  that  financial  differential  is  necessary  to  attract  and 
sustain  effective  online  instructors  as  successful  online  faculty  equate  one  online  course  as  equivalent  to 
two  grounded-based  courses  in  relation  to  resources  needed  to  insure  high  quality  and  meaningful  learning 
for  students. 

Pedagogical  training  should  emphasize  accepted  best  practices  for  online  education.  Specifically, 
faculty  should  receive  training  in: 

• Facilitating  interaction  and  discussion  in  online  courses; 

• Facilitating  active  learning  and  collaborative  online;  and 

• Assessment  and  evaluation  for  online  courses. 

Some  portion  of  either  the  technical  or  pedagogical  training  for  faculty  to  teach  online  should  be  delivered 
online  so  that  faculty  experience  online  education  from  the  student  point  of  view.  Finally,  faculty 
mentoring  is  critical  to  online  teaching.  Without  assistance  for  colleagues  and/or  the  department  of 
Distance  Feaming,  it  is  doubtful  for  successful  online  education. 
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Evolution  in  the  Classroom: 

What  Teachers  Need  to  Know  about  the  Video  Game  Generation 


Elizabeth  S.  Simpson 
University  of  Wyoming 

Research  driving  the  mandates  of  the  current  education  reform  law,  No  Child  Left  Behind, 
indicates  a 300%  increase  during  the  last  10  years  in  students  being  labeled  with  specific  learning 
disabilities.  In  addition  there  has  been  a dramatic  increase  in  the  number  of  minority  students  labeled  as 
having  learning  and  emotional  disabilities  (U.S.  Department  of  Education,  2002).  One  possible  reason  for 
this  disturbing  evidence  is  that  our  teachers  and  schools  are  not  prepared  to  assess  the  skills  or  meet  the 
needs  of  a “new”  generation  of  learners  entering  their  classrooms.  The  new  generation  I am  referring  to  are 
those  students  who  have  daily  access  to  interactive  3D  environments,  spend  a significant  amount  of  time 
exploring  that  environment  and  who  have  the  skills  to  maneuver  and  problem  solve  within  it.  That 
environment  is  the  3D  world  of  video  games.  There  is  evidence  indicating  the  same  students  who  are  most 
at  risk  for  failure  in  the  traditional  classroom  setting,  spend  an  average  of  twenty-seven  minutes  per  day 
more  than  their  counterparts  using  video  games  (National  Center  for  Educational  Statistics,  2002: 
Woodward,  2002).  David  Sousa,  author  of  the  book,  "How  the  Special  Needs  Brain  Learns",  feels  the  way 
our  schools  are  currently  set  up  could,  in  fact,  be  disabling  for  some  students  who,  given  a different 
learning  environment,  might  otherwise  thrive.  Although  research  on  the  subject  is  still  in  its  formative 
years,  the  research  led  by  Henry  Jenkins,  Kurt  Squire,  Marc  Prensky  and  James  Gee,  lends  strong  support 
to  Sousa’s  beliefs. 

The  students 

Kids  today  spend  more  time  outside  the  classroom— exploring,  questioning  and  problem  solving 
than  they  do  “learning”  in  school.  They  make  decisions  based  on  their  every  day  experiences,  their 
interests,  their  strengths,  and  their  desires.  Their  world  expects  them  to  interact  with  it  and  when  they  do  so, 
they  are  in  command.  They  welcome  a challenge  and  the  opportunity  to  test  new  boundaries.  They  are  in 
control  of  their  own  destiny  and  the  heroes  of  their  own  adventures.  They  support  and  guide  the  direction  of 
a billion  dollar  industry.  These  are  the  gamers,  the  video  game  generation  (Beck  & Wade,  2004).  Most  are 
bright  and  inquisitive  and  can  do  things  with  a computer  that  the  average  teacher  would  never  even  attempt. 
Yet,  many  of  the  gamers  in  school  today  are  at  risk  for  failure.  The  risk  is  not  predominantly  for  the  reasons 
commonly  sited,  i.e.,  socio-economic  status,  cultural  difference,  disability  or  lack  of  opportunity.  The 
gamers  are  not  formally  labeled.  Instead  they  are  referred  to  as  lazy,  apathetic,  behavior  problems,  truant, 
disengaged  or  suffering  from  a bad  attitude.  Simply  put,  they  are  not  happy  in  school.  They  are  bored.  They 
aren’t  challenged.  They  see  no  relevance  in  the  subject  matter.  Some  are  dropping  out  and  many  just  are 
waiting  it  out,  apathetic  and  unengaged.  I believe  it  is  because  of  the  video  game. 

Like  it  or  not,  video  games  play  a daily  part  in  the  lives  of  almost  every  child,  either  through 
actual  game  play  or  through  having  friends  and  family  who  are  gamers.  Therefore,  understanding  how  this 
new  generation  perceives  the  world,  interacts  and  problem  solves  within  it  can  have  a serious  impact  on 
how  well  the  video  game  generation  performs  in  school.  “Once  the  child  reaches  the  age  of  six  or  seven, 
the  influence  of  the  culture— whether  or  not  it  is  manifested  in  a school  setting— has  become  so  pervasive 
that  one  has  difficulty  envisioning  what  development  could  be  like  in  the  absence  of  such  cultural  supports 
and  constraints.”  (Gardner,  1991,  p.  105).  The  basis  for  the  gamers’  cultural  and  social  connections  is  often 
a shared  gaming  experience.  Video  games  cross  all  cultural  and  ethnic  boundaries.  Not  recognizing  that 
these  shared  experiences  exist  can  deeply  affect  the  way  this  generation  responds  to  public  education. 
Teachers,  who  have  never  shared  these  experiences,  face  a deep  chasm  when  trying  to  communicate  with 
this  increasing  segment  of  their  classroom  population. 

The  facts 

• 92%  of  children,  ages  2-17  play  video  and  computer  games  (National  Institute  on  Media  and  the 

Family,  2001). 

• Video  games  are  most  prevalent  among  children  who  are  of  elementary  school  age  and  older 

(Woodward  & Gridina,  2000). 
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• The  average  American  child  grows  up  in  a home  with  two  televisions,  three  tape  players,  three 
radios,  two  VCRs,  two  CD  players,  one  video  game  player  and  one  computer  (Kaiser  Family 
Foundation,  2005). 

• 60%  of  all  Americans,  or  about  145  million  people,  play  interactive  games  on  a regular  basis 
(Entertainment  Software  Association,  2004). 

• From  1997  to  2001,  computer  ownership  has  increased  37%  for  Whites,  53%  for  Blacks,  53%  for 
Hispanics,  38%  for  Asian/Pacific  Islanders,  and  46%  for  Native  Americans  (National  Center  for 
Educational  Statistics,  2003). 

• According  to  parents,  children  between  the  ages  of  2 and  17  spend  almost  6 V2  hours  a day  in  front 
of  electronic  screens  either  television,  video  games  or  a computer  (Woodward  & Gridina,  2000). 

• Approximately  80%  of  American  families  own  a computer  and  78%  have  video  game  equipment 
in  their  homes.  (National  Center  for  Educational  Statistics,  2003;  National  Institute  on  Media  and 
the  Family,  2001). 

• Households  with  more  than  one  child  are  more  likely  to  have  a video  game  system  than  those  with 
only  one  child.  Homes  with  at  least  one  boy  are  more  likely  to  have  a video  game  system  (76%  of 
homes  with  at  least  one  boy  vs.  58%  of  homes  with  only  girls)  (Woodward  & Gridina,  2000). 

• 100%  of  the  college  students  polled  in  a recent  survey  conducted  at  27  campuses  have  played  a 
video  game  either  on  a console  or  on  the  computer  (Wilkens,  2004). 

• Parents  are  the  driving  force  behind  the  video  game  industry.  In  2004,  over  50%  of  parents  polled 
said  they  were  going  to  buy  their  child  a videogame  for  Christmas.  90%  of  all  videogames  are 
purchased  by  individuals  18  years  or  older  (Entertainment  Software  Association,  2004). 

The  teacher’s  world — who  is  in  the  driver’s  seat? 

The  majority  of  today’s  teachers  are  women  (79%)  with  an  average  age  of  46.  In  some  states,  60% 
of  the  teachers  are  over  50  (National  Center  for  Educational  Statistics,  2003).  Most  of  the  experience  these 
teachers  have  had  with  technology  is  limited  to  word  processing,  data  bases,  presentation  software,  and 
perhaps  some  multimedia  such  as  video  and  digital  cameras.  When  these  teachers  bring  technology  into  the 
classroom,  they  bring  what  they  are  familiar  with,  which,  although  is  a step  in  the  right  direction,  it  is  not  in 
itself  engaging  for  the  students.  Teachers  do  not  typically  see  the  videogame  as  a learning  tool.  The 
teachers  are  working  within  an  environment  where  change  tends  to  be  slow,  money  scarce,  and  bureaucracy 
plentiful.  Unfortunately  their  students  are  living  in  an  environment  where  change  is  rapid,  constant,  and 
anticipated.  This  generation  of  learner  wants  to  be  challenged.  They  want  to  have  some  control  over  the 
choices  they  make  and  the  direction  they  take.  The  students  are  ready  to  be  in  the  driver’s  seat  and  in  the 
fast  lane. 

The  problem  facing  teachers  is  that  they  do  not  want  to  give  up  the  wheel.  Their  lack  of 
understanding  of  this  new  generation  leads  them  to  believe  that  if  they  give  up  the  wheel,  the  car  will  crash. 
The  opposite  may  be  true.  The  number  one  request  by  teachers  for  professional  development  revolves 
around  behavior  management/control  issues.  Teachers  are  encouraged  to  embrace  differentiated  instruction, 
a method  of  meeting  the  needs  of  the  individual  learner,  yet  many  teachers  are  unsure  how  to  achieve 
differentiation  while  still  maintaining  control,  thus  differentiation  is  slow  to  be  integrated  into  the  system. 
Differentiation  is  inherent  in  the  video  game  generation’ s world.  They  make  choices  on  a daily  basis  in 
every  venue  from  food  choices,  information  to  be  gathered,  games  to  play,  products  to  buy,  shows  to  watch 
to  cell  phone  rings,  music,  friends,  styles,  the  list  is  endless.  If  they  want  something  they  don’t  have  all  they 
have  to  do  is  “Google”  it  and  they  are  presented  with  a menu  of  choices  of  where  to  get  it,  and  how  much  it 
will  cost,  the  history  of  the  item  and  related  subjects  and  sites.  The  environment  is  always  individualized 
for  them.  This  is  the  “have  it  your  way”  generation.  The  “do  it  my  way”  philosophy  of  the  traditional  old 
school  does  not  work  for  this  generation. 

The  gamer’s  world — Move  over,  I’m  driving,  buckle  up! 

The  game  generation  is  in  charge  and  ready  to  lead.  Beck  and  Wade  (2004)  point  out  in  their 
timely  book,  Got  Game:  How  the  Gamer  Generation  Is  Reshaping  Business  Forever , that  there  are  several 
distinct  influences  the  experience  of  playing  video  games  has  on  the  on  the  way  the  game  generation  learns 
and  organizes  information.  Prensky  (2004)  has  likened  the  skills  of  the  gamer  to  that  of  the  highly  effective 
workers  of  Stephen  Covey’s  era.  Gamers  believe,  for  example: 

There  is  always  an  answer 
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Video  Games  are  basically  fair.  There  is  always  a problem(s)  that  has  a solution(s)  which  lead  to 
an  end  result— the  object  of  the  game.  There  can  be  many  different  routes  to  reach  the  solution,  all  of  which 
count  equally  if  you  achieve  the  goal.  The  answer  is  rarely  obvious.  A correct  answer  will  give  you 
information  which  will  be  useful  in  reaching  the  goal,  thus  you  must  persevere  to  find  a correct  answer. 

The  answer  is  always  relevant.  You  might  be  frustrated  for  a while  and  you  may  need  help  in  finding  it,  but 
it  is  always  there.  Cheats  (hints)  are  built  into  the  program  and  are  part  of  the  resources  available  to  you. 
Cheats  are  OK  because  you  are  learning  (gaining  valuable  information)  as  you  move  forward  toward  the 
goal.  In  schools,  the  answer  is  given  to  you,  it  is  often  not  linked  to  anything  relevant,  there  is  only  one 
right  answer  and  one  right  way  to  get  there  and  cheats  are  not  to  be  tolerated!  Students  rarely  if  ever 
associate  fairness  with  schools. 

Nothing  is  impossible 

In  any  game,  you  have  the  tools  and  the  talent  to  be  successful  on  your  own  or  you  may  connect 
with  someone  who  has  the  information  you  need  in  order  to  move  forward  (collaboration).  You  see 
yourself  and  your  friends  do  amazing  things  such  as  save  the  world  from  terrorists  or  alien  invasions,  beat 
the  best  team  in  the  NFL,  create  thriving  civilizations  and  manage  a successful  small  business.  You  have 
the  power  to  control  your  destiny.  You  can  accomplish  anything  you  want,  and  therefore,  you  are 
motivated. 

Trial  and  error 

It  is  proven  through  game  experience  that  this  is  the  fastest,  most  efficient  way  to  learn.  If  trial  and 
error  does  not  work,  you  know  where  to  find  the  necessary  resources  and  you  can  access  them  at  will.  You 
are  in  control  of  your  own  success  or  failure.  If  you  persevere,  you  will  achieve  your  goal.  If  you  do  not 
win,  restart  and  try  again.  You  will  not  make  the  same  mistakes  twice.  Failure  is  a learning  experience,  not 
an  end  result  as  it  is  so  often  in  schools. 

Competition  and  collaboration 

Competition  is  inherent  in  game  structure.  Gamers  are  always  competing.  Competition  is  the 
motivating  factor.  Competition  does  not  eclipse  collaboration;  in  fact,  collaboration  is  often  an  integral  part 
of  furthering  your  success.  Competition  and  collaboration  are  symbiotic  rather  than  mutually  exclusive 
concepts. 

Roles  are  clear 

In  games,  roles  are  clearly  defined.  You  are  the  droid  or  the  Jedi,  the  good  guy  or  the  bad  guy,  an 
employee  or  the  boss.  You  choose  your  role  and  understand  its  limitations.  You  understand  the  rules,  the 
tools  at  your  disposal  and  you  are  willing  to  take  the  risks.  In  schools,  the  roles  are  not  as  clear. 

Gamers  are  autonomous 

Merri am- Webster  Online  Dictionary  (http://www.m-w.com/)  defines  autonomous  as  the  quality  or 
state  of  being  self-governing,  especially  the  right  of  self-government.  The  culture  of  gamers  is  very  much  a 
culture  of  personal  and  professional  autonomy.  Gamers  play  well  with  others  but  accept  responsibility  and 
consequences  for  their  actions,  both  positive  and  negative,  within  the  game.  Gamers  feel  they  have  the  right 
to  choose  their  own  path  and  are  confident  in  exercising  that  right. 

Gamers  dominate  their  culture 

There  is  little  or  no  attention  paid  to  elders.  Most  elders  are  non-gamers.  Non-gamers  don’t  have  a 
clue  what  you  are  doing  and  therefore,  they  can’t  be  of  any  help.  Most  gamers  have  come  to  this  conclusion 
through  trial  and  error.  Edginess  and  attitude  are  dominating  elements  of  the  culture.  Gamers  are  risk 
takers.  They  move  fast  and  play  hard.  They  are  the  stars  in  their  own  adventure.  They  are  responsible  for 
their  own  success.  Their  experience  tells  them  that  with  patience  and  perseverance,  they  will  succeed. 

Do  your  own  thing: 

Within  the  game  generation,  leaders  are  irrelevant  and  often  evil;  ignore  them.  This  is  the  attitude 
of  the  dot.com  generation.  They  created  and  built  the  eBusiness  concept  that  dominated  the  economic 
landscape  during  the  1990’ s.  At  the  time,  there  were  no  leaders  to  follow  in  this  field.  They  carved  their 
own  niche  and  the  payoff  was  huge.  They  did  their  own  thing.  When  many  of  the  dot.com’ s collapsed,  they 
pressed  “RESTART.”  They  learned  from  their  mistakes  and  most  have  become  rising  executives  in  new 
endeavors. 

The  games  vs.  school 

So,  what  is  it  about  video  games  that  make  gamers  respond  to  the  world  differently  from  the  way 
they  respond  in  the  classroom?  Compared  to  the  classroom,  games  are  empowering,  motivating, 
individualized  differentiated  learning  environments  with  set  rules  which  value  the  efforts  of  the  individual 
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child.  Games  are  challenging  and  motivating.  They  offer  the  child  a shared  experience  with  their  peers  in  a 
collaborative  environment.  They  are  a platform  for  problem  solving.  The  structure  is  apparent,  the  rules  are 
clear  and  unambiguous,  and  your  role  in  the  game  is  well  defined.  The  goal  is  always  attainable. 

Video  games  are  rule-based 

The  rules  are  applied  equally  to  each  player.  The  rules  of  the  game  have  to  be  sufficiently  well 
defined  so  there  is  no  room  for  individual  interpretation.  Consequences  of  player  behavior  are  clearly  either 
positive  or  negative.  If  there  is  a disagreement  about  the  rules  of  the  game  the  game  is  stopped  until  the 
disagreement  has  been  resolved.  In  schools  we  tend  to  consider  our  classrooms  rules  based.  However,  the 
teacher  is  the  keeper  and  the  interpreter  of  the  rules.  The  teacher  chooses  the  game.  The  teacher  makes  the 
rules  and  can  change  the  rules  at  will,  either  for  the  class  or  for  the  individual.  The  game  continues  even 
when  rules  are  broken.  Gamers,  who  are  confronted  with  rules  that  are  arbitrary  and  subjective,  will  often 
“shut  down”  and  refuse  to  play  the  teacher’s  game. 

Video  games  offer  various  routes  to  success 

School  has  a set  outcome  with  one  acceptable  route.  If  you  stay  in  school  for  12  or  more  years, 
you  will  get  out.  Schools  are  currently  set  up  to  sort  students  by  age,  skill  is  secondary.  For  example,  a 
student  who  is  very  successful  at  his/her  grade  level  will  stay  at  that  grade  level  and  not  move  up.  However, 
if  you  are  not  successful  in  your  grade  level,  you  will  fall  back  a level.  Regardless  of  your  success  in  the 
lower  level,  you  will  not  be  able  to  move  back  up. 

Skill  is  not  relevant  to  upward  movement. 

In  a video  game,  success  at  one  level  catapults  you  to  the  next,  more  challenging  level.  Usually, 
there  are  several  different  routes  available  to  reach  the  next  level.  These  routes  are  based  on  individual 
skill.  You  may  fall  back  temporarily,  but  there  is  always  a route  that  will  allow  you  to  move  back  up  to  the 
next  level  and  beyond. 

Player  effort  influences  the  game  outcome 

The  amount  of  energy  the  player  puts  into  the  game  invests  the  player  with  the  outcome.  Teachers 
are  frustrated  with  the  lack  of  effort  students  are  putting  into  their  assignments  and  coursework.  Yet  these 
are  the  same  students  who  spend  hours  playing  games  which  they  find  relevant,  challenging  and  fun. 

The  player  is  attached  to  the  outcome 

The  player  has  an  emotional  attachment  to  the  outcome,  meaning  the  player  is  happy  if  they  win  or 
unhappy  if  they  loose.  Juul  (2003)  calls  this  the  “game  contract,”  which  the  player  agrees  to  when  they  play 
the  game.  The  spoilsport  is  one  who  refuses  to  seek  enjoyment  in  winning,  or  refuses  to  become  unhappy 
by  loosing  in  which  case,  the  gamer  is  not  attached  to  the  goal.  Many  gamers  are  not  sure  what  the  goal  is 
in  school  or  how  they  can  help  accomplish  that  goal.  For  this  generation  of  learner,  that  is  a serious 
problem.  Gamers  often  fail  to  see  relevance  in  the  information  presented  in  the  classroom.  In  games,  the 
objective  is  clear— build  an  empire,  save  the  world,  kill  the  zombies.  The  gamer  sees  value  in  killing 
zombies  and  saving  the  world.  The  gamer  often  does  not  see  value  in  filling  out  their  spelling  notebook, 
completing  another  math  worksheet  or  reading  a 15th-century  author.  Teachers  have  to  understand  how 
gamers  learn  in  order  to  make  the  content  relevant. 

Video  games  simulate  real-life  consequences 

Medical  colleges  use  video  games  to  train  surgeons,  the  Air  Force  uses  “virtual  flight  simulators” 
to  train  its  fighter  pilots  and  many  major  corporations  are  using  “virtual  models”  to  train  their  junior 
executives.  If  you  screw  up,  you’re  patient  dies,  you  crash  or  you’re  fired.  These  are  real-life  consequences. 
Games  can  often  mirror  the  complexities  of  real  life.  Gamers  view  the  world  from  multiple  viewpoints. 
They  play  games  from  the  first,  second  and  third  person  perspective,  opening  different  routes  to  problem 
solving.  They  can  literally  get  into  a character’s  skin  and  see  the  world  from  the  character’s  point  of  view. 
They  understand  that  problems  are  complex  but  given  a relevant  goal,  clear  expectations  and  the  autonomy 
to  utilize  a variety  of  resources,  they  can  and  do  find  the  solution. 

The  bad  news 

Game  over 

Educators  have  been  slow  to  pick  up  on  this  fundamental  shift  in  the  way  the  video  game 
generation  learns.  They  know  something  is  very  different  in  the  classroom,  they  just  can’t  pin  point  what  it 
is.  Teachers  are  frustrated  by  their  inability  to  connect  with  students.  This  dissatisfaction  is  evidenced  by 
the  63,000  open  teaching  positions  across  the  nation  in  2003.  The  current  majority  of  teachers  is  over  the 
age  of  46  and  began  teaching  before  the  video  game  was  such  a big  part  of  the  youth  culture.  The  average 
teacher  rarely  shares  the  same  daily  experiences  or  have  the  same  interests  as  their  students  whose  native 
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experience  with  technology  is  so  different  from  their  own.  Students  and  teachers  are  in  a constant  battle  for 
control  and  influence  based  on  opposing  expectations.  There  is  a huge  cultural  divide  between  the  video 
game  generation  and  the  teachers  and  that  divide  is  taking  its  toll  on  both  students  and  teachers  alike. 

Students  are  telling  teachers  loud  and  clear  that  they  don’t  want  to  play  the  teacher’s  game 
anymore— game  over.  We  are  dealing  with  a new,  rapidly  growing  culture  that  refuses  to  be  force  fed  a 
“canned  education.”  Apathetic  students  are  willing  to  be  in  school,  perhaps,  but  motivated  only  by  their 
parents,  their  friends  or  the  law.  Even  though  they  are  physically  in  attendance,  they  are  not  engaged  in  the 
“game”  we  call  school.  This  attitude  is  a slap  in  the  face  to  our  traditional  educational  system,  but  it  is  a 
fact  and  it  is  not  going  to  go  away.  It  should  be  a wake  up  call— the  rules  have  changed. 

Given  that  100%  of  college  students  polled  on  27  campuses  indicated  they  had  played  a 
videogame  in  the  last  12  months,  we  can  assume  that  gamers  are  not  a rarity  in  the  classroom.  Today’s 
dominant  emerging  culture  is  the  culture  of  gamers.  Thirty-nine  percent  are  women  (Entertainment 
Software  Association,  2004).  The  students  of  the  gamer  generation  are  our  future  CEO’s,  doctors,  lawyers, 
scientists  and  world  leaders.  Beck  and  Wade  (2004)  conducted  a large  scale  study  of  2,500  business 
professionals  to  determine  whether  the  experience  of  gaming  and  growing  up  surrounded  by  games, 
changes  attitudes,  expectations,  and  abilities  related  to  how  the  video  game  generation  performs  in  the 
business  world  (p.21).  The  results  indicate  that  gamers  see  the  world  very  differently  than  do  their  parents, 
teachers  or  other  non-gamers.  The  structure  of  the  game  molds  the  gamers’  experiences,  leading  to  a 
different  way  of  looking  at  the  world  and  given  a certain  situation,  determining  how  to  best  interact. 
Teachers  who  are  not  gamers  do  not  live  in  the  same  world  and  therefore  cannot  see  the  possibilities.  The 
gamers  are  trying  to  send  a message  to  their  teachers  and  to  the  educational  system  as  a whole.  Clearly, 
teachers  are  going  to  have  to  rethink  how  they  present  material  in  the  classroom. 

Embrace  the  video  game 

Many  commercially  available  video  games  can  be  used  as  learning  tools  in  the  classroom.  Current 
research  has  identified  the  main  features  of  video  games  that  teachers  need  to  be  aware  of  when  using  them 
in  the  classroom  (Juul,  2003;  McFarlane,  Sparrowhawk  and  Herald,  (2002).  Lesson  plans  using  existing 
video  games  as  the  learning  platform  are  rapidly  emerging.  Squire  has  created  teaching  materials  to  support 
the  learning  in  Sid  Meir’s  game,  Civilization  (Squire,  2003).  Teaching  with  video  games  can  open  new 
avenues  of  communication  between  teachers  and  students.  They  offer  the  student  a familiar  environment  in 
which  to  demonstrate  their  skills,  to  move  to  the  next  level  (Hostetter,  2002  Prensky,  2004).  Teachers  who 
are  familiar  with  the  covert  learning  found  in  video  games  can  create  win-win  opportunities  in  the 
classroom.  Ninety  percent  of  all  elementary  and  secondary  students  polled  use  their  home  computers  for 
videogames,  while  only  eighty-three  percent  use  them  for  school  assignments  (National  Center  for 
Educational  Statistics,  2003.)  Why  not  merge  the  two.  I am  not  saying  that  the  video  game  should 
completely  replace  the  traditional  methods  of  teaching.  There  are  certain  foundations  that  should  stay 
intact.  I am,  however,  saying  that  the  use  of  video  games  as  a teaching  tool  deserves  serious  consideration 
as  a means  of  presenting  information  and  bridging  learning  concepts.  We  must  give  teachers  permission 
meet  the  unique  learning  needs  of  the  gamer  generation.  Today’s  teachers,  and  especially  those  just 
entering  the  profession,  need  to  partner  with  the  students  and  get  on  board  -the  students  are  already  there. 
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Why  do  I need  to  learn  this  if  I can  get  it  all  online? 

Current  issues  in  the  education  of  teacher-librarians. 

Andrew  J.  M.  Smith 
Edward  C.  Lomax 
Georgia  State  University 

There  have  long  been  arguments  in  the  library  community  over  the  necessity  of  retaining  courses 
such  as  cataloging  in  library  science  degrees.  The  arguments  for  doing  away  with  such  courses  appear  to  be 
even  stronger  in  the  school  library  media  community.  With  the  ever-increasing  availability  of  cataloging 
records  from  “one-stop”  vendors  there  are  seemingly  fewer  and  fewer  opportunities  for  media  specialists  to 
put  their  cataloging  expertise  into  practice.  In  addition,  cataloging  has  traditionally  been  the  “statistics” 
class  of  all  library  programs  - with  the  reputation  of  being  difficult,  unapproachable,  and  consequently 
feared  by  students.  Online  vendors  now  offer  not  only  instant  cataloging,  but  instant  collection  analysis, 
instant  collection  development,  instant  weeding,  and  so  on.  Now  that  students,  apparently,  do  not  need  to 
have  these  skills,  they  are  asking  how  these  courses  are  still  relevant  to  them. 

Other  traditional  foundational  courses  are  also  being  questioned,  including  courses  in  reference 
services.  Students  are  again  questioning  the  usefulness  of  reference,  especially  as  it  relates  to  print  based 
resources,  when  so  much  information  is  available  quicker  on  the  Internet. 

To  what  extent  does  this  questioning  reflect  a real  need  for  change  within  preparation  programs 
for  school  library  media  specialists?  Are  we  simply  clinging  on  to  content  because  this  is  what  has  always 
been  done  and  we  are  loath  to  sacrifice  any  content,  as  this  will  in  some  way  diminish  our  stature  as  a field? 
Can  we  justify  the  continued  study  of  these  subjects?  Are  they  foundational  to  everything  we  do  as  library 
media  specialists  or  must  we  reappraise  our  whole  philosophy  of  what  a library  media  specialist  is?  A good 
way  to  clarify  the  issue  is  to  examine  the  case  of  cataloging,  as  this  is  the  course  where  faculty  often 
experience  the  most  resistance. 


The  necessity  of  teaching  cataloging 

“Cataloging  is  hard.  As  a library  media  specialist,  I’ll  never  do  cataloging:  Why  I do  need  to  learn 
this  stuff?”  - overheard  during  a class  break 

“Cataloging  is  the  intellectual  foundation  of  librarianship — it  is  the  way  in  which  good  librarians, 
in  all  fields,  think.”  - Michael  Gorman 

There  are  a number  of  reasons  why  budding  library  media  specialists  would  find  a cataloging 
course  useful  in  the  professional  work  they  are  being  educated  and  trained  to  do.  Although  school  libraries 
are  sometimes  seen  as  simply  a repository  for  books,  in  fact  they  are  fast  becoming  storehouses  of 
increasingly  complex  information  packages  that  are  multimodal  in  their  orientation,  organization,  and  use. 
Often  these  resources  feature  a number  of  authors  associated  with  these  works  and  these  resources  are 
frequently  multidisciplinary  in  nature.  Another  trend  in  American  schools  that  has  a direct  impact  on  the 
school  library,  is  the  increasing  cultural  and  ethnic  diversity  of  the  learning  community.  Each  of  these 
groups  comes  into  the  school  library  and  its  information  resources  with  a unique  perspective  that  has  been 
formed  by  its  culture,  intellectual  ability,  and  educational  level.  The  increasing  diversity  of  information 
resources,  particularly  networked  information  resources,  and  information  technologies  that  enable  access  to 
these  resources  require  that  we  media  specialists  and  our  learning  communities  be  conversant  in  both  the 
design  and  capabilities  of  these  resources  in  order  to  make  the  best  use  of  these  resources.  As  media 
specialists,  if  our  aim  is  to  provide  the  best  service  to  our  learning  communities,  we  need  to  be 
knowledgeable  of  the  way  that  information  is  created,  manipulated,  displayed,  and  used  (Carpenter  et  al, 
1991;  Intner,  2002). 


Cataloging  education  and  its  role  in  school  librarianship 

The  library  exists  as  a repository  for  recorded  knowledge,  as  a knowledge  portal,  and,  particularly 
in  the  school  setting,  as  an  access  point  to  networked  information  resources.  In  a recent  opinion  on  the 
subject  of  the  core  competencies  required  of  the  librarian,  the  Task  Force  on  Core  Competencies  of  the 
American  Library  Association  (2005)  made  the  following  statement  of  principle: 
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The  ability  to  organize  collections  of  informational  materials  in  order  that  desired  items  can  be 
retrieved  quickly  and  easily  is  a librarian’s  unique  competency.  Well-organized  collections  are  the 
foundation  for  all  library  service.  Competence  in  organizing  collections  involves  thorough  knowledge  of 
bibliographic  and  intellectual  control  principles  and  standards,  understanding  of  how  to  apply  these 
principles  and  standards  in  practical,  cost-effective  operations;  and,  the  ability  to  collaborate  with  those 
who  provide  systems  for  managing  organizational  functions  such  as  library  vendors  and  institutional 
computer  center  staff  members.  (American  Library  Association,  2005,  p.  1) 

Cataloging  and  indexing  are  two  of  the  principal  means  of  organizing  information  in  a way  that  is 
useful  for  information  seekers  (Holley,  2002).  As  Gorman  (2002)  suggests,  “cataloging  is  not  just  for 
catalogers”  (p.  10).  For  those  prospective  library  media  specialists  and  others  who  believe  that  knowledge 
of  bibliographic  control  has  no  place  in  the  education  of  the  library  media  specialist,  they  should  be 
mindful  that  the  key  to  the  effective  use  of  the  online  public  access  catalog,  a fixture  in  their  media  center, 
online  databases,  and  Google,  is  an  understanding  of  “the  way  in  which  recorded  knowledge  and 
information  is  organized  for  retrieval. ...”  (Gorman,  2002,  p.  2).  Gorman  also  argues  that  librarians  have  a 
thorough  knowledge  of  cataloging  standards  and  principles  are  essential  to  all  aspects  of  library  work  (e.g., 
Reference,  Library  Instruction,  Circulation,  Collection  Development,  and  Cataloging)  and  in  order  for  them 
to  be  effective  librarians.  “Librarians  will  be  more  effective  in  finding  information  both  for  themselves  and 
for  their  patrons  if  they  understand  the  structure  of  records  in  the  online  catalog,  the  principles  of  authority 
control  including  cross  references,  and  the  MARC  format  insofar  as  it  affects  indexing  (Holley,  2002,  p. 
45).  Given  that  school  library  users  employ  online  public  access  catalogs  and  online  databases 
independently  in  their  search  form  information  in  lieu  of  the  face-to-face  interview  with  the  reference 
librarian,  school  librarians  can  employ  this  greater  understanding  to  their  advantage  by  creating  more 
accurate  cataloging  records  and  indexed  databases  to  support  their  learning  communities  in  their  quest  for 
useful  information  and  knowledge. 


Why  do  we  need  to  learn  this  stuff? 

Although  the  arrival  of  the  Internet  and  the  cornucopia  of  networked  information  resources 
available  using  this  medium  and  its  many  information  services  has  changed  the  face  of  knowledge  creation 
and  information  retrieval  forever,  the  mission  of  the  school  librarian  remains  the  same:  to  organize 
information  for  the  benefit  of  the  learning  community;  to  develop  a collection  of  useful  information 
resources  for  this  community,  and  to  provide  effective  instruction  and  instructional  services  that  have  the 
potential  to  benefit  this  community  from  cradle  to  grave.  As  Gorman  (2002)  states,  “I  believe  that 
cataloging  is  a central  part  of  that  mission  in  ways  both  obvious  and  subtle,  and  that  we  must  devise  ways 
in  which  education  for  our  profession  understands  and  communicates  that  centrality”  (p.  13).  Cataloging  is 
not  merely  a peripheral  activity,  one  that  we  must  do  in  order  to  get  materials  on  the  shelves  so  that  the  real 
work  can  get  done.  Rather  it  is  (or  should  be)  the  very  foundation  of  all  our  efforts  to  support  learning  and 
to  build  an  information  literate  population. 

The  fault  of  the  profession? 

Perhaps  one  answer  is  that  we  are  doing  a poor  job  in  educating  our  students  as  to  the  power  and 
foundational  nature  of  these  courses.  While  the  internet  is  indeed  a rich  resource  of  information,  it  has  now 
reached  a stage  where  the  lack  of  organizational  elements  within  it  have  resulted  in  the  ready  acceptance  of 
poor-quality  or  inaccurate  information,  simply  because  it  is  easy  to  find.  Older  Internet  users  come  to  on- 
line resources  with  a thorough  training  in  the  searching  and  evaluation  techniques  they  honed  in  the  print 
world.  In  other  words  they  have  a thorough  understanding  of  the  practice  and  principles  of  information 
literacy.  In  particular  they  understand  the  importance  of  various  organizational  devices:  of  cataloging,  of 
indexing  and  of  headings,  summaries  and  tables  of  contents.  The  students  in  today’s  K-12  environment 
may  be  technologically  knowledgeable,  but  they  are  often  lacking  these  basic  information  processing  tools. 
If  our  cataloging  and  reference  courses  are  truly  foundational  to  information  literacy,  then  we  should  have 
little  problem  in  justifying  them  to  our  students.  However,  we  must  not  only  make  the  case  for  this,  but 
communicate  this  effectively  to  our  students.  We  must  also  ensure  that  our  courses  themselves  support  the 
new  information  technologies,  and  that  they  are  relevant,  not  in  the  sense  that  they  throw  out  the  old  and 
embrace  only  the  new,  but  in  that  they  are  truly  comprehensive  and  are  taught  in  enough  depth  and  breadth 
to  include  the  new  technologies.  It  may  be  prudent  to  reexamine  our  basic  library  courses  to  ensure  we  are 
not  promoting  suboptimal  skills. 
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The  need  for  balancing  our  preparation  programs 

A second  major  topic  concerns  those  areas  that  are  currently  lacking  in  library  media  preparation 
programs.  As  we  move  away  from  the  idea  of  the  “book  warehouse  keeper”  to  the  “teacher-librarian”  there 
is  increasing  pressure  put  upon  programs  to  give  their  students  adequate  preparation  in  the  teaching  aspects 
of  the  job.  While  some  states  require  that  school  library  media  specialists  be  certified  teachers  before  they 
become  media  specialists,  this  is  by  no  means  a universal  requirement.  In  many  ways  preparation  programs 
that  accept  non- teachers  have  a tough  assignment,  in  that  they  still  have  a similar  number  of  credit  hours, 
but  those  must  be  divided  between  teaching  courses  and  library  courses.  Even  those  courses  that  only 
accept  teachers  may  find  that  teachers  who  were  trained  some  years  before  are  not  as  well  prepared  in  areas 
such  as  technology  as  could  be  desired.  Over  ten  years  ago  various  authorities  raised  the  issue  of  the 
teaching  component  of  the  teacher-librarian  post  (Amey,  1993:  Amey  & Reesa,  1993).  Concern  was  raised 
about  the  nature  of  our  preparation  programs,  as  to  whether  we  were  truly  educating  teacher  librarians,  or 
simply  training  them.  As  Altmann  (quoted  in  Amey,  1993,  (][  2)  states: 

Teacher-librarians  too  often  see  themselves  simply  as  conduits  of  information,  or  perhaps  even  for 
knowledge,  identifying  and  passing  on  useful  materials  to  their  students  and  fellow  teachers.  We 
have  not  taught  them  to  recognize  complexity,  to  assess  themselves  as  factors  in  their  teaching,  to 

reflect  critically  on  the  materials  and  techniques  they  work  with.  How  to  educate,  rather  than  train, 
teacher-librarians  is  a major  problem  in  an  environment  in  which  too  little  time  is  available  for 
professional  education  in  this  field. 

However,  Hamilton  (2001)  raises  an  interesting  counter  argument  to  this  from  the  Canadian  perspective 
where  he  suggests  that  it  is  the  emphasis  on  teacher-librarian  that  has  diminished  the  role  of  the  teacher- 
librarian  in  the  schools  and  has  marginalized  the  librarianship  aspects  of  the  job.  He  also  laments  the  fact 
that 


Overwhelming  technology 

A third  area  of  concern  regarding  the  preparation  of  library  media  specialists  is  the  growth  in 
general  technology  within  schools  and  the  need  for  technology  support.  Increasing  demands  are  being 
placed  on  media  specialists  to  act  as  technology  coordinators.  Many  schools  are  no  longer  retaining  full- 
time technical  support  staff  for  computer  labs  and  equipment,  but  are  shifting  these  duties  to  the  media 
specialists.  This  has  the  effect  of  giving  the  media  specialists  three  major  tasks:  maintaining  and 
administering  the  media  center  and  its  collections,  teaching  information  literacy  skills  and  collaborating 
with  other  teaching  faculty,  and  providing  technical  support,  training  and  expertise. 

There  are  two  disturbing  aspects  to  this.  Firstly  we  are  not  preparing  our  students  to  be  technical 
support  in  the  way  that  schools  now  require  it.  Our  teacher-librarians  are  being  trained  in  the  infusion  of 
technology  into  the  curriculum.  Technology  classes  focus  on  the  use  of  technology  as  a tool  for  learning, 
and  give  students  basic  knowledge  of  a wide  range  of  software  programs  that  can  be  of  great  value  to  both 
students  and  teachers.  These  courses  do  not  generally  cover  software  or  hardware  trouble  shooting,  network 
administration,  or  the  kind  of  data  gathering  and  administration  that  are  properly  the  responsibility  of 
technical  support  personnel  and  data  professionals. 

Secondly  our  teacher-librarians  are  being  required  to  spend  increasing  amounts  of  time  away  from 
the  educational  and  library  duties  they  are  trained  to  perform,  to  work  as  technical  troubleshooters  and 
support  personnel.  Obviously  the  more  time  spent  in  basic  technology  support  and  service,  the  less  time  is 
available  for  information  literacy  instruction,  collaboration  with  teachers,  and  therefore  poorer  educational 
services  from  the  media  center,  at  least  in  quantity  if  not  quality.  It  is  easy  to  see  how  this  could  quickly 
result  in  dissatisfaction  with  the  media  center  program,  its  decline  in  importance  and  the  subsequent 
allocation  of  more  technical  support  duties  to  the  media  specialist. 

A call  for  communication 

One  answer  to  these  problems  may  be  increased  communication.  Firstly  we  must  agree  amongst 
ourselves  as  to  the  usefulness  of  the  various  components  of  our  preparation  programs.  We  should 
encourage  communication  between  library  schools  and  colleges  of  education  so  that  we  may  present  a 
united  front  on  what  a teacher-librarian  should  be.  If  both  teaching  and  librarian  skill  sets  are  truly  needed, 
then  we  must  work  together  to  promote  this,  rather  than  allowing  each  profession  to  devalue  the  work  of 
the  other. 

Secondly  we  must  communicate  effectively  with  our  students.  They  must  not  only  be  told  the 
value  of  the  skills  and  education  they  receive,  but  we  must  provide  courses  and  experiences  that 
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demonstrate  this,  so  they  may  have  the  opportunity  to  value  their  preparation  and  understand  how  each  part 
works  to  support  the  others. 

Thirdly  we  must  educate  the  educators:  teachers,  principals,  and  superintendents.  We  know  the 
valuable  contribution  our  teacher-librarians  can  make  to  encouraging  and  ensuring  student  success,  but 
obviously  large  numbers  of  teachers  and  school  administrators  do  not.  We  must  also  prepare  our  students  to 
educate  their  schools  as  to  the  significant  enhancements  they  can  offer  to  every  learning  environment.  They 
must  be  their  own  advocates  but  they  must  be  given  the  tools  and  skills  necessary  to  do  this. 

The  field  of  teacher-librarianship  offers  essential  components  to  the  creation  of  effective  learning 
environments.  However,  it  can  only  do  so  if  its  practitioners  are  allowed  to  do  participate  in  a meaningful 
way.  It  is  our  responsibility  to  promote  this  by  equipping  our  students  to  the  best  of  our  ability.  This  will 
require  all  of  us,  whether  in  traditional  library  schools  or  colleges  of  education,  to  agree  on  the  necessity  of 
both  traditional  library  and  education  curricula  and  to  advocate  strongly  for  the  continuation  of  the  library 
media  specialist  within  our  schools. 
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Don't  tell  them  to  read  the  instructions  -Tell  them  what  to  do! 

How  low  reading  adult  learners  have  a better  chance  to  "get  it"  when 
their  instructional  software  narrates  to  them. 

H.  Solomon 

Abstract 

Once  a weakness  in  text- enablement  has  been  identified  in  a student,  giving  him  or  her  a 
multimedia  software  package  that  starts  off  with  a demand  to  read  immediately  identifies  the  software  to 
the  user  as  reading-based.  This  often  means  to  the  user  that  it  is  an  opportunity  to  work  in  an  area  of  a 
learning  modality  weakness.  The  multimedia  capacity  of  today's  desktop  computers  makes  possible  design 
opportunities  that  are  no  longer  restricted  to  text  as  the  principle  learning  modality.  Narration  of 
instructions  can  greatly  enrich  the  learning  experience  for  low  text-enabled  users,  making  it  more  likely 
that  desired  learning  outcomes  will  occur.  This  presentation  reports  on  a research  project  that  tracked  user 
proficiency  in  applying  a learned  procedure  after  instruction  in  text  and  narrated  modes.  A post-hoc 
analysis  of  student  performance  indicated  that  providing  only  the  reading  modality  significantly  increases 
the  likelihood  of  failure  to  learn  the  procedure  well  enough  to  complete  it. 

Creating  instructional  software  that  includes  narration  takes  more  development  time,  as  sound 
clips  need  to  be  created,  synched  to  the  onscreen  visual  information,  and  brought  under  user  control.  This  is 
clearly  more  effort  than  is  often  put  into  text-based  products.  Including  narration  would  slow  the  rate  at 
which  software  products  could  be  produced  and  increase  the  cost  of  producing  the  software.  So,  the 
question  arises  about  whether  the  benefits  of  including  narration  outweigh  the  additional  costs. 

Rose  and  Meyer  (2002),  in  advocating  standards  of  universal  design  for  learning  (UDL),  suggest 
that  instructional  media  needs  to  be  flexible  because  the  learners  have  different  needs  and  capabilities  in 
different  areas.  What  works  for  one  doesn't  necessarily  work  for  another.  They  go  on  to  elaborate  that  text 
is  the  dominant  modality  in  schools,  which  is  only  beneficial  for  the  students  who  are  good  at  using  text. 

For  those  who  don't  use  text  very  well,  however,  software  should  compensate  and  accommodate.  The 
accommodations,  while  they  may  be  more  costly  to  create,  are  often  used  in  unanticipated  ways  that  add 
value  to  the  software  beyond  what  was  expected  at  the  time  of  the  original  design.  They  liken  this  to  the 
architectural  principle  of  Universal  Design  which  calls  for  architectural  accommodations  for  universal 
access  that  wind  up  being  used  extensively  by  those  who  were  not  thought  to  be  the  immediate 
beneficiaries. 

Shih  and  Alessi  (1996),  in  the  light  of  an  experimental  finding  showing  no  learning  effect  among 
college  senior  undergraduate  students,  cautioned  against  the  development  of  fancy  multimedia  programs 
that  don't  have  any  learning  effect.  They  noted  that  it  was  possible  that  voice  may  have  some  benefits 
including  increased  realism  and  naturalness,  better  comprehension  by  poor  readers,  less  distraction  of 
visual  attention,  increased  engagement,  and  better  synching  to  temporal  events.  In  working  with  their 
chosen  experimental  population  however,  advantages  of  text  over  voice,  such  as  greater  flexibility  of 
processing  rate,  more  than  compensated  for  these  differences. 

Some  experimental  evidence  supports  combining  sound  with  other  media.  Lai,  (2000)  for 
instance,  found  a learning  effect  resulting  from  the  combination  of  audio  and  animation  where  the  observed 
learning  effect  was  explained  as  the  audio  drawing  student  attention  to  the  dynamic  graphic  and  explaining 
the  visual  image  step  by  step.  Nocente  (1996),  found  an  effect  in  favor  of  audio-enhanced  instruction  for 
12th  grade  trigonometry  students  in  spite  of  the  audio  being  a redundant  repetition  of  onscreen  text  Along 
the  way,  she  also  noted  that  students  appreciated  the  option  of  being  able  to  turn  the  audio  off. 

A study  (Solomon,  2005)  was  conducted  at  a community  college  in  the  southeast  United  States 
that  investigated  the  effectiveness  of  different  modalities  for  presenting  information  by  randomly  assigning 
students  to  learn  a procedure  from  a multimedia  software  tutorial  in  three  different  versions.  One  version 
favored  a reading  presentation  modality,  one  a spoken  presentation  modality,  and  one  a choice  between  the 
two  modes.  The  outcome  measured  was  the  proficiency  of  student  performance  of  a PowerPoint  procedure 
shown  in  the  tutorial.  Except  for  the  mode  of  presentation,  all  information  contained  in  all  versions  was 
identical.  Students  who  volunteered  for  the  study  were  recruited  from  a population  enrolled  in  a special 
program  for  students  who  had  been  admitted  despite  low  entrance  exam  scores  under  the  college's  open 
admissions  policy. 
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Although  no  significant  difference  between  the  proficiency  of  student  performances  could  be 
attributed  to  the  treatments,  analysis  of  the  collected  data  revealed  that  a much  higher  than  anticipated 
number  of  the  students  who  were  unable  to  complete  the  procedure  successfully  had  been  assigned  to 
complete  the  reading  modality  software.  This  indicates  that  for  a poorly  text-enabled  population, 
presentation  of  instructions  for  a procedure  in  text  mode  is  likely  to  result  in  failure  to  learn  the  procedure. 
Introduction: 

Cognitively  complex  procedural  skills  could  be  described  as  those  that  require  a learner  to  follow 
a multi-step  path  beyond  the  threshold  of  tasks  with  which  they  are  previously  familiar.  This  is  a relativistic 
definition  of  such  skills,  and  encompasses  the  notion  that  tasks  diminish  in  cognitive  complexity  after  many 
repetitions.  Consider,  for  instance,  the  creation  of  a presentation  in  Microsoft’s  PowerPoint  software.  For 
many  people  attending  the  AECT  convention,  this  task  is  often  preceded  by  the  word  “just”  as  in  “just 
make  up  your  PowerPoint”.  For  many  other  persons,  the  thought  of  undertaking  this  task  rouses  trembling 
and  severe  fear  of  failure. 

If  we  think  back  to  the  time  when  mastering  this  task  was  a challenge  to  us,  we  probably  relied 
extensively  on  printed  manuals,  tutorial  programs,  in-program  help  features,  and  other  ways  to  find  out 
what  we  needed  to  do  to  get  over  completing  the  task  a first  or  second  time.  A fortunate  few  of  us  were 
coached  through  our  first  few  times  by  some  other  person  who  had  previously  mastered  the  task,  and  was 
well  practiced  at  its  procedure.  Those  of  us  without  the  service  of  a coach  were  very  likely  to  have 
encountered  text-based  media  like  the  words  printed  on  pages  of  manuals  or  appearing  on  computer 
screens.  For  most  of  us,  the  use  of  text-based  media  posed  no  strenuous  challenge  because  reading  these 
words  was  a skill  so  regularly  practiced  that  it  offered  no  difficulty.  We  were  able,  because  of  the  extent  to 
which  we  automate  the  task  of  reading,  to  concentrate  our  attention  on  mastering  the  tasks  involved  with 
using  the  program. 

For  persons  who  have  had  difficulty  with  reading,  and  who  have  not  automated  reading  tasks  to 
the  same  extent  as  have  the  professors  and  graduate  students  attending  this  conference,  the  ability  to 
concentrate  on  a task  may  be  reduced  by  a need  to  concentrate  on  successful  reading.  This  makes  the  task 
of  learning  the  procedure  more  difficult.  In  such  cases,  it  is  reasonable  to  suggest  that  replicating  the 
services  of  the  coach  by  providing  narrated  directions  for  performing  the  procedure  might  increase  the 
likelihood  that  the  learners  will  not  only  learn  but  also  become  more  fluent  with  the  procedure. 

Many  different  phenomena  have  been  given  the  name  "technology  gap,"  ranging  from  the  inability 
of  the  American  urban  poor  to  access  computer  networks  to  third  world  lack  of  information  infrastructure 
to  parents  not  understanding  the  appropriate  technological  innovations  to  supply  for  their  children  at  the 
beginning  of  a school  term.  Recent  experiences  of  this  author  in  working  with  firefighter  training  indicate 
a gap  of  substantial  nature  in  the  training  methods  and  tools  used  by  instructors  from  different  types  of 
organizations.  Training  organizations  that  work  with  persons  in  traditionally  blue-collar  trades  are 
periodically  expected  to  update  training  materials  to  meet  new  goals  and  standards.  While  training 
produced  may  rely  on  traditional  classroom  settings,  even  here,  high  tech  presentation  is  increasingly 
expected.  Training  is  expected  to  be  delivered  in  ways  in  which  the  trainees  have  a greater  chance  to  "get 
it". 

While  it  is  possible  to  ask  trainees  to  read  through  instructions,  reading  is  only  one,  and  certainly 
not  the  only,  means  to  acquire  procedural  knowledge.  The  most  commonly  used  alternative  means  is 
listening.  In  numerous  studies  (Rehaag  and  Szabo  (1995),  Holmes,  (1985))  acquisition  of  knowledge  by 
reading  is  shown  to  be  more  rapid  than  the  acquisition  of  knowledge  by  listening.  Yet  it  stands  to  reason 
that  listening  would  have  advantages  for  those  who  read  slowly.  If  learning  software  is  designed  without 
allowing  the  slow  readers  to  hear  instead  of  read,  that  software  potentially  widens  the  academic 
achievement  gap  between  good  and  poor  readers.  For  instance,  if  students  are  able  to  calculate  the  answers 
to  math  problems  but  have  difficulty  reading  the  directions  for  proceeding  through  more  complex  problem 
forms,  they  are  likely  to  be  considered  poor  in  math.  The  design  of  the  learning  experience  gives  reading 
difficulties  the  potential  to  mask  ability  in  other  areas. 

Although  technological  enhancements  of  learning  materials  are  common,  the  inclusion  of  narrated 
directions  is  far  less  widespread.  There  is  a relative  lack  of  research  documenting  any  gains  made  by 
learners  when  listening  is  offered  as  an  alternative  to  reading.  Researchers  attempting  to  conduct  such 
research  encounter  a number  of  difficulties.  Included  in  the  difficulties  are  such  factors  as  difficulty  with 
recruiting  appropriate  populations  and  with  standardizing  the  instruction  received  so  that  identical  learning 
content  is  presented  in  all  experimental  conditions.  To  illustrate  the  first  of  these,  consider  that  the  strong 
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preponderance  of  studies  conducted  in  college  classrooms  results  in  much  research  that  uses  populations 
pre-selected  on  the  basis  of  the  abilities  tested  as  part  of  admissions  standards  - including  reading. 

An  advantage  of  printed  as  compared  with  narrated  presentation  is  that  the  printed  material  lingers 
in  front  of  the  reader  over  time  and  can  be  approached  at  a pace  dictated  by  the  reader  rather  than  the 
narrator.  Passages,  or  parts  of  passages,  may  be  reviewed  by  the  reader  as  required  to  insure  confidence  in 
understanding  the  content.  Narration  is  not  as  natural  as  reading  when  it  is  repeated.  While  reading  allows 
the  reader  to  select  the  amount  of  redundancy  that  will  produce  the  best  available  learning  experience  for 
himself  or  herself,  no  equivalent  selection  method  is  available  to  support  listening. 

In  a sense,  any  study  comparing  learning  that  results  from  printed  instead  of  audited  presentation 
is  bound  to  be  confounded  by  the  modes  representing  different  bundles  of  affordances.  Consider  that  the 
following  characteristics  inhere  in  each  of  the  two  learning  modalities. 


Table  2.  Comparison  of  Affordances  of  Reading  and  Listening 


Afforded  by  reading 

Afforded  by  listening 

Faster  maximum  rate  of  acquisition 

Reduced  load  of  visual  acquisition 

Learner  is  in  control  of  presentation  pace 

Available  in  more  lighting  conditions 

Faster  access  to  different  parts 

Sense  of  hearing  is  always  on 

No  limits  on  reviewing 

Eyes  free  to  concentrate  on  visual  information 

While  table  1 is  incomplete,  it  is  sufficient  to  demonstrate  that  the  question  of  which  modality  is 
better  for  learning  is  not  likely  to  be  settled  in  any  one  study.  For  a study  to  reasonably  compare  narrated 
and  textual  presentation  modalities  for  learning,  it  must  make  an  attempt  to  offset  the  natural  advantage  of 
either  learning  mode,  and  be  conducted  with  a population  for  whom  reading  acquisition  and  listening 
acquisition  rates  could  be  presumed  to  be  approximately  equal. 

One  reason  that  a comparison  study  makes  sense  now  is  that  by  mixing  the  technological  capacity 
of  computers  with  carefully  created  instructional  design,  it  becomes  possible  to  offset  the  inherent 
advantages  of  these  learning  modalities.  For  instance,  the  faster  maximum  rate  of  acquisition  advantage  for 
reading  might  be  offset  with  the  presentation  of  reading  material  as  a sequence  of  one-sentence  units 
displayed  with  a sequence  of  mouse.  This  would  impose  a series  of  physical  clicking  actions  between 
sentences  and  slow  down  the  rate  of  reading.  The  action  of  exposing  the  next  sentence  would  take  about  as 
much  time  as  a narrator’s  natural  pauses  for  commas  and  periods.  Table  2 below  indicates  a set  of 
compensations  that  could  equalize  a study  for  the  natural  advantages  appearing  in  table  1 . 

Table  3.  Methods  for  offsetting  the  Affordances  of  Reading  and  Listening 


Advantage 

Favors 

Offset  by: 

Faster  maximum  rate 

Reading 

Triggered  sequential  presentation  of  text 

Control  of  presentation  pace 

Reading 

Introduce  learner  triggers  to  control  pace  in  both 
modes 

Faster  access  to  different  parts 

Reading 

Segment  presentation  and  tightly  index  segments  for 
review 

No  limits  on  reviewing 

Reading 

Allow  unlimited  audio  replay  of  segments  under 
learner  control. 

Reduced  load  of  visual 
acquisition 

Listening 

Increase  percentage  of  screen  space  that  is  visual 
representation  of  the  procedure  being  learned. 

Immunity  to  poor  lighting 
conditions 

Listening 

Keep  all  visual  attention  on  a monitor  in  light- 
controlled  conditions 

No  earlids 

Listening 

Conduct  studies  during  normal  waking  hours 

Freedom  from  visual  distraction 

Listening 

Remove  irrelevant  visual  information. 

Of  those  items  in  table  2 that  are  supposed  to  compensate  for  the  advantages  of  listening,  most  of 
them  require  little  more  than  attention  to  detail.  The  only  exception  is  increasing  the  percentage  of  screen 
space  that  is  visual  representation  of  the  procedure  being  learned.  By  choosing  a procedure  performed  on  a 
computer  as  the  task  to  be  learned,  the  learning  environment  provided  by  the  computer  can  be  made  to  look 
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like  the  actual  environment  in  which  the  task  will  be  performed.  This  strategy  reduces  the  number  of 
potential  visual  distractions  by  reducing  the  number  of  visual  items  on  the  screen. 

In  addition  to  the  triggered  sequential  presentation  of  text  described  earlier,  other  design 
characteristics  that  tend  to  reduce  the  advantage  of  reading  do  so  by  enhancing  the  flexibility  of  the 
listening  modality  to  make  it  more  the  equivalent  of  reading.  Introducing  triggers  to  control  the  pace  of 
presentation  is  already  a familiar  tool  that  characterizes  just  about  all  PowerPoint  demonstrations.  Being 
able  to  replay  specific  short  sequences  of  narrated  clips  on  demand  is  a design  feature  that  adds  to  the 
narrated  modality  without  decreasing  the  ability  to  repeat  a section.  This  is  like  being  able  to  turn  to  a 
selected  page  of  an  audio  book.  Shih  and  Alessi  (1996)  proposed  this  type  of  audio  repeatability  as  a way  to 
better  synch  audio  presentations  with  any  visuals  being  described.  By  now,  such  commonplace 
technologies  as  DVDs  have  built  similar  capabilities  into  products  widely  distributed  in  home 
entertainment  systems.  As  a result,  learners  do  not  have  to  learn  to  use  this  capacity.  They  just  need  to  be 
made  aware  that  it  is  available. 


The  Study 

The  study  that  provides  the  basis  for  this  paper  was  conducted  by  its  author  to  measure  procedural 
task  fluency  increases  for  low-reading  community  college  students  when  narration  replaced  text  as  the 
medium  in  which  directions  were  made  available.  The  study  was  conducted  at  a community  college  in  the 
Southeast  United  States  in  October  of  2004.  Participants  in  the  study  came  from  two  student  populations. 
The  first  and  larger  group  (n  = 69)  was  identified  by  the  institution  as  “low  reading”  on  the  basis  of  low 
entry  examination  scores  on  the  CPT  test.  The  second  group  (n  = 18)  served  the  function  of  being  an 
experimental  control,  and  was  recruited  from  individuals  enrolled  in  an  introductory  educational 
technology  class.  Both  groups  volunteered  to  participate  in  the  study  and  were  induced  to  do  so  by  a 
promise  of  extra  credit  in  one  class  in  which  they  were  enrolled. 

A task  was  selected  that  would  be  new  to  the  participants  so  that  there  was  no  previous  familiarity 
with  it.  The  selected  task  involved  the  use  of  a procedure  in  PowerPoint  of  adding  narrated  audio  clips  to 
different  zones  of  a background  illustration.  A multimedia  computerized  tutorial  was  devised  that 
demonstrated  the  procedure.  The  tutorial  concluded  with  a question  asking  the  participants  whether  they 
had  grasped  the  procedure.  If  a positive  response  was  received,  the  program  opened  a specially  prepared 
PowerPoint  file  that  was  ready  to  have  the  procedure  implemented  as  many  times  as  possible  within  fifteen 
minutes.  The  maximum  number  of  times  that  the  task  could  have  been  completed  was  thirteen.  None  of  the 
participants  successfully  completed  the  task  the  maximum  available  number  of  times. 

The  tutorial  was  developed  in  two  different  forms.  In  each,  the  participants  advanced  at  their  own 
pace  through  a series  of  14  screens  in  which  the  procedure  to  be  used  was  discussed  and  demonstrated.  As 
different  parts  of  the  procedure  were  shown,  the  screens  used  were  taken  from  screen  shots  of  PowerPoint 
as  it  would  appear  at  the  currently  demonstrated  stage  of  the  procedure.  The  areas  of  the  screen  that  would 
be  used  by  the  actual  program  were  activated  to  produce  what  looked  like  the  same  result  that  the  actual 
program  would  give  in  response  to  the  same  user  action.  The  difference  between  the  two  versions  of  the 
tutorial  was  limited  as  far  as  possible  to  the  presentation  of  directions  for  doing  the  steps  of  the  tutorial.  In 
the  text-based  version,  the  directions  were  presented  as  text  appearing  in  a consistent  location  on  the 
screen,  informing  the  user  how  to  proceed.  In  the  narrated  version,  the  directions  were  presented  as  a series 
of  spoken  audio  clips  while  no  text  appeared  on  the  screen.  The  same  words  were  used  in  both  versions. 

Strategies  discussed  above  for  reducing  the  inherent  advantage  of  reading  as  a presentation 
modality  were  incorporated  in  the  design  of  the  tutorial  software.  Although  a direct  comparison  was  the 
goal  of  the  study,  a third  version  of  the  tutorial  software  in  which  the  choice  between  modalities  was  made 
available  to  students  as  a menu  item  was  also  included.  For  this  version,  a portion  of  the  introduction  to  the 
software  was  enlarged  by  the  inclusion  of  a timed  screen  (10  seconds)  showing  the  program  user  how  to 
switch  between  modes  if  he  or  she  felt  a need  to  change  from  one  mode  to  another.  This  version  was 
developed  to  allow  this  study  to  provide  potential  confirmation  of  previous  studies  (Nocente,  1996)  that 
indicated  that  choice  of  modality  resulted  in  superior  learning  performance.  It  also  facilitated  a potential 
comparison  between  randomly  selected  and  self-selected  participant  groups  on  other  performance 
parameters. 

For  the  study,  the  research  questions  were: 

1)  What  is  the  effect  on  performance  of  a procedure  when  students  learn  from  narrated  rather  than 

textually  presented  multimedia  software? 
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2)  Is  there  a differential  learning  effect  for  low  reading  ability  learners  of  learning  a procedure 
from  narration  rather  than  text? 

3)  If  narration  is  optional  and  under  user  control,  will  learners  with  low  reading  ability  choose  to  use  it? 

Based  on  a review  of  literature  concerned  with  similar  studies,  the  hypothetical  positions  adopted  for 
investigation  were: 

1)  Students  who  learn  a procedure  from  narrated  rather  than  textually  presented  multimedia 
software  will  perform  the  procedure  correctly  at  a higher  rate. 

2)  Among  low  reading  ability  learners,  the  effect  of  learning  from  narration  will  be  more 
pronounced  than  it  will  be  for  high  reading  ability  learners. 

3)  If  narration  is  optional  and  under  user  control,  learners  with  low  reading  ability  will  choose  to 
use  it  for  most  learning  situations. 

The  Study’s  Results 

The  results  of  the  study  showed  no  significant  difference  in  the  fluency  with  which  the  learned 
procedure  was  performed  in  favor  of  any  of  the  treatment  conditions.  While  the  observed  rank  order  of 
mean  performance  fluencies  was 

1)  offering  a choice  of  modality 

2)  offering  instruction  in  only  the  listening  modality  and 

3)  offering  instruction  in  only  the  reading  modality, 

no  significant  advantage  was  found  in  favor  of  the  listening  modality  treatment  on  the  ability  to  perform 
procedures  by  students  whose  reading  scores  were  either  above  or  below  the  median  reading  level. 

Even  in  the  light  of  further  analysis  that  included  a post  hoc  breakdown  of  the  participants  by  CPT 
score  quartile,  there  remained  no  significant  advantage  of  the  listening  modality.  Low  reading  students  who 
participated  in  the  study  did  not  show  a significant  tendency  to  choose  the  listening  modality  version  of  the 
software.  While  they  might  have  been  able  to  avoid  the  potential  for  reading  difficulty  associated  with  a 
choice  of  the  reading  modality  version,  low  reading  students  in  this  study  exhibited  no  clear  desire  to  take 
advantage  of  the  choice  offered. 

The  Study’s  Post-Hoc  Result 

While  the  study  failed  to  confirm  its  hypotheses,  post  hoc  statistical  analysis  of  its  results  led  to 
this  paper.  Collected  data  may  sometimes  be  used  in  ways  that  go  beyond  its  support  of  the  original 
research  hypotheses  of  a study.  In  this  case,  the  search  for  explanations  of  why  the  hypotheses  were  not 
supported  by  the  data  was  as  revealing  as  the  results  of  the  actual  study. 

If  learners  going  through  a learned  procedure  accomplish  it  a greater  number  of  times  in  a fixed 
interval,  they  may  be  said  to  be  more  fluent  with  the  procedure.  The  difference  between  completing  the 
procedure  3 times  or  6 times  in  a 15-minute  period  could  be  attributed  to  greater  fluency.  A fluency  score 
of  0,  on  the  other  hand,  indicates  that  the  learner  “didn’t  get  it.”  Embarrassing  as  it  is  for  designers  of 
tutorial  software  to  admit,  sometimes  student  will  fail  to  learn.  And  failure  to  learn,  instead  of  being  treated 
as  an  aberration,  ought  to  be  of  interest  to  us  if  it  is  associated  with  any  one  particular  treatment. 

Analysis  of  the  data  revealed  that  students  offered  only  the  reading  version  of  the  tutorial  software 
scored  a larger  than  expected  number  of  zeroes.  This  meant  that  there  was  a high  likelihood  that  students 
were  more  likely  to  fail  to  learn  to  perform  the  procedure  demonstrated  when  they  had  been  assigned  to  a 
text-based  presentation.  Of  the  seventeen  students  who  completed  the  study  but  were  unable  to  complete 
any  labels,  ten  were  from  the  "reading  only"  treatment  group.  A binomial  test  of  this  phenomenon  against 
the  expected  value  of  .33  (28  of  85)  showed  that  the  possibility  of  this  distribution  of  zero  scores  occurring 
by  chance  was  less  than  .05.  It  seems  then,  that  for  the  population  of  this  study,  assigning  instruction  only 
available  as  text  significantly  increases  the  likelihood  of  failure.  Table  3 shows  the  results  of  the  binomial 
analysis. 

Table  3:  Binomial  Analysis  of  Distribution  of  Zero  Scores  by  treatment 


Number 

Proportion 

Proportion 

Treatment 

of  Zeroes 

of  Total 

of  Zeroes 

Sig. 
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Reading  Modality 

10 

.33 

.61 

.025 

Listening  Modality 

3 

.39 

.18 

.055 

Choice  Modality 

4 

.28 

.21 

.460 

Total 

17 

1.00 

1.00 

This  distribution  of  zero  scores  strongly  suggests  that  among  low  reading  community  college 
students,  greater  than  average  difficulty  will  be  encountered  learning  procedures  when  the  structure  of  the 
lesson  mandates  that  they  learn  from  reading.  Given  the  dominance  of  low  reading  levels  in  the  population 
tested,  an  alternative  interpretation  of  this  phenomenon  is  that  the  listening  modality  helps  students  learn 
the  procedure  well  enough  to  complete  it. 


Implications 

Presuming  the  veracity  of  the  study  reported  about  in  this  paper,  the  conclusion  may  be  reached 
that,  of  community  college  students  with  previously  noted  low  reading  ability,  providing  text  based 
instruction  in  procedures  involving  computer  programs  increases  the  chance  of  their  failure  to  master  the 
procedures  described.  This  conclusion  is  clearly  limited  in  its  power  by  the  low  number  of  study 
participants  whose  score  was  zero.  Because  tutorial  software  isn’t  designed  to  produce  failures,  conclusions 
drawn  on  the  way  a limited  number  of  failures  line  up  can  only  be  supported  by  studies  with  very  small 
samples.  In  a low  reading  population,  where  failure  to  duplicate  the  shown  procedure  is  more  likely  to 
occur  than  average,  only  17  of  the  89  participants  could  be  included  in  a sample  of  “zero”  scores.  Because 
only  around  20%  of  a trait- selected  population  would  meet  the  criteria  for  being  included  in  the  sampled 
group,  studies  of  this  sampled  group  require  much  larger  base  populations  than  the  one  mentioned  here. 

On  first  glance,  it  would  appear  that  only  a small  number  of  persons  fall  into  the  category  of  those 
who  may  avoid  failure  in  procedural  learning  by  being  exposed  to  tutorials  as  narration  rather  than  as  text. 
And  because  the  result  is  only  related  to  procedures  performed  using  a computer,  it  may  be  tempting  to 
suggest  that  not  many  low-reading  adults  are  likely  to  be  involved  in  computer  usage,  thus  minimizing  the 
extent  of  the  problem.  However,  a US  Bureau  of  Labor  Statistics  report  from  October,  2003,  on  the  type  of 
computer  activity  at  work  by  selected  characteristics  claims  that  workers  with  less  than  a high  school 
education  were  likely  to  be  using  a computer  in  43%  of  available  jobs.  In  addition,  the  Alliance  for 
Excellent  Education  (2003)  makes  reference  to  a report  by  the  National  Association  of  Manufacturers  that 
in  2001,  more  than  $60  billion  was  spent  on  training,  and  that  much  of  that  was  on  remedial  reading, 
writing,  and  mathematics.  (Alliance  for  Excellent  Education,  2003)  The  combination  of  the  two  reports 
provide  a strong  indication  that  failure  to  learn  procedures  on  a computer  because  the  presentation  of 
information  is  in  a mode  that  leads  to  some  learner  failures  is  a problem  worthy  of  solution. 

When  people  undertake  new  tasks,  their  success  with  the  tasks  can  depend  on  the  instruction  they 
receive  about  doing  them.  If  we  want  to  challenge  low  readers  to  new  accomplishments  in  non-reading 
related  areas,  it  makes  sense  to  provide  an  alternative  to  reading  text  in  the  form  of  listening  to  narration. 
Making  this  option  available  decreases  the  chances  of  failure. 

The  time  that  this  study  can  have  an  impact  on  how  instruction  is  designed  is  rapidly  approaching. 
High  bandwidth  internet  delivery,  which  is  becoming  increasingly  widespread,  is  making  the  transfer  of 
recorded  spoken  instruction  a reality.  File  formats  for  the  transfer  of  sound-related  recordings  are  already  in 
place  and  popular.  Podcasting  is  within  a few  years  of  becoming  as  ubiquitous  as  radio.  It  is  easy  to 
visualize  the  transfer  of  the  instructions  for  steel  cellar  door  assembly,  for  instance,  into  an  iPod-like  device 
worn  at  the  point  of  assembly.  The  cost  to  produce  sound-based  instructional  devices  is  dropping 
dramatically.  Users  will  have  already  absorbed  the  costs  of  playback  units.  The  time  spent  in  the  production 
of  a narration  of  instructions  is  little  more  than  the  time  it  takes  to  have  a proofreader  speak  them  aloud. 

One  advantage  audio  has  over  text  is  that  it  is  weightless  and  not  bound  to  any  particular  place.  It 
is  easy  to  summon  up  another  scene  in  which  you  have  brought  your  car  to  your  local  auto  dealer’ s service 
department  to  replace  the  disk  brake  pads  of  your  front  wheels.  The  service  writer  has  looked  up  the  year 
and  model  of  your  vehicle  and  typed  in  the  kind  of  job  to  be  done.  A computer  in  the  service  office  has 
downloaded  a set  of  audio  files,  each  a narrated  subroutine  of  the  overall  brake  pad  replacement  procedure. 
The  mechanic,  equipped  with  a set  of  ear  buds  connected  by  a wireless  connection  to  a miniature  receiver 
unit  in  a specially  made  shirtsleeve  pocket,  removes  the  device  and  touches  a stylus  to  the  “getting  ready” 
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menu.  This  contains  a submenu  of  items  including  preparing  the  work  area,  tools  you’ll  need,  etc.  The 
touch  of  the  stylus  to  the  “tools  you’ll  need”  area  brings  up  the  sound  of  a master  mechanic’s  voice  telling 
her  that  in  addition  to  the  toolbox  of  conventional  hand  tools,  she  will  also  need  a brake  adjustment  wrench 
and  a spring  tension  gauge. 

The  mechanic  selects  the  “performing  the  work”  menu,  which  reveals  a choice  of  several  sub- 
procedures arranged  in  the  order  in  which  they  are  normally  performed.  She  selects  the  “removing  the 
calipers”  item  and  the  master  mechanic  tells  her  about  the  special  caliper  removal  edge  built  into  the 
wrench  she  brought  with  her  earlier.  She  holds  the  tool  in  the  position  described,  moves  it  to  the  location 
described,  pushes  in  the  direction  described,  and  confirms  visually  that  the  proper  clearance  for  caliper 
release  is  achieved. 

Once  she  has  the  caliper  removed,  she  sees  a few  parts  whose  removal  procedure  is  obvious. 
Having  replaced  brake  pads  many  times,  she  doesn’t  need  to  approach  the  menu  sequentially.  Instead,  she 
quickly  works  through  the  parts  of  the  procedure  she  remembers  until  she  gets  to  the  “pad  guide  slot  for 
removal”  item.  At  this  point,  she  picks  up  on  the  master  mechanic’s  narration  again.  Overall,  she  spends 
about  10  minutes  on  the  procedure  for  this  wheel.  The  “standard  jobs”  credit  she  is  getting  for  it  is  20 
minutes.  Similar  savings  occur  on  the  other  side  of  the  car  as  well. 

Key  to  her  success  is  the  presence  of  just-in-time  instruction  structured  in  a way  that  lets  her 
concentrate  on  what  she  sees  and  what  to  do  with  the  tools  and  parts  on  which  she  is  working.  Key  to  the 
just-in-time  instruction  is  a smattering  of  off-the-shelf  computer  connected  products  and  the  network  of 
dealerships  sponsored  by  the  manufacturer.  The  scenario  in  question  costs  little  more  to  put  into  place 
beyond  the  already  existing  high  bandwidth  network  than  the  will  to  put  the  files  into  the  right  databases. 

With  the  imaginary  mechanic  in  this  scenario,  menu  reading  was  used,  but  the  bulk  of  the 
delivered  instruction  took  the  form  of  narration.  If  the  instruction  had  been  in  the  form  of  text,  the  number 
of  words  or  pages  of  instruction  that  would  have  to  be  carried  would  have  been  enormous.  To  this  day, 
service  manuals  are  heavy,  inaccessible,  sloppy  attention  hogs.  They  rarely  deliver  exactly  the  information 
desired,  and  in  the  few  cases  that  they  do,  font  sizes  become  small  enough  to  result  in  eyestrain. 

One  implication  of  the  results  of  the  study  at  hand  is  that,  in  circumstances  where  technology 
facilitates  narratively-delivered  instruction  about  procedures,  that  it  is  worthwhile  to  do  so.  Consider  for  a 
moment  the  main  result  of  this  study  - that  there  was  no  significant  difference  in  fluency  of  procedure 
performance  that  could  be  attributed  to  any  of  the  modalities  in  which  the  instruction  was  delivered.  This, 
in  effect,  says  that  text-based  procedural  instruction  did  not  enable  readers  to  become  any  more  fluent  with 
procedures  than  did  listening.  The  delivery  of  exclusively  text-based  instruction  cannot  be  supported  on 
grounds  of  learning  efficiency,  cost,  or  convenience.  The  only  reason  left  for  its  dominance  is  that  “we’ve 
always  done  it  this  way.” 

The  study  at  hand  cannot  say  that  a menu  of  audio  files  is  the  most  effective  method  of  bringing 
instruction  to  the  point  of  need.  The  procedure  for  which  the  “concentration  of  failures”  phenomenon  was 
observed  was  limited  to  one  done  on  computers.  This  has  not  been  shown  here  to  translate  into  workshop  or 
pencil  and  paper  settings.  But  there  is  good  reason  to  expect  that  similar  results  can  be  obtained.  One 
reason  for  this  expectation  is  the  match  between  procedures  and  audio  of  their  both  being  extended  into 
time.  Text  cannot  claim  the  same  match  because  it  locates  information  at  a place.  In  doing  so,  it  creates  a 
need  on  the  part  of  learners  to  alternately  focus  visual  attention  to  multiple  places.  The  service  manual  and 
the  textbook  both  match  on-screen  text  fields  in  adding  visual  clutter  to  the  learner’s  area  of  visual 
concentration.  The  results  of  the  study  at  hand  simply  indicate  that  additional  research  about  the  value  of 
narratively  delivered  instruction  needs  to  be  done. 

The  imaginary  mechanic  scenario  is  one  of  the  possible  areas  in  which  auditory  delivery  of 
information  makes  sense.  It  depicts  a combination  of  a workplace,  a motivated  adult  learner,  and  sequential 
procedures.  The  study  at  hand  dealt  with  a school  setting,  a computer-based  task,  cooperative  adult  learners 
with  varying  levels  of  motivation,  and  a sequential  procedure.  Room  for  research  exists  in  changing  the 
settings,  changing  the  ages  of  the  learners,  and  changing  the  task  to  one  where  multiple  paths  to  successful 
completion  exist.  It  should  be  clear  from  the  scenario  described  that  there  will  be  places  where  auditory 
delivery  of  instruction  reduces  errors,  improves  advance  preparation,  and  increases  efficiency.  It  is 
reasonable  to  expect  that  procedural  learning  is  an  arena  in  which  auditory  instruction  can  be  expected  to  be 
of  benefit. 

There  will  be  some  areas  where  learning  through  narration  is  cumbersome  and  inefficient. 
Although  “talking  books”  are  enjoyed  by  many  listeners  at  many  different  reading  levels,  they  typically 
take  longer  to  complete  because  the  speaker’s  rate  of  delivery  is  less  than  a reader’s  rate  of  reading. 
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Literature,  then,  seems  to  be  most  efficiently  delivered  in  text-based  media  forms.  History  may  be  another 
area  where  auditory  presentation  of  information  is  slower  and  less  flexible  than  reading.  In  areas  like  this, 
little  of  what  is  expected  of  a learner  is  to  be  able  to  perform  a procedure.  Often,  the  expectation  is  that  the 
learner  be  able  to  perform  the  act  of  test- taking  in  the  aftermath  of  his  or  her  learning.  Because  the  usual 
form  of  “test-taking”  places  the  test  taker  in  front  of  visually  presented  text  and  then  results  in  a 
performance  of  creating  or  modifying  a visually  presented  document,  textual  presentation  is  closer  to  the 
way  the  task  is  assessed  than  is  auditory  presentation.  This  is  a better  alignment  of  the  assessment  to  the 
presentation  method,  which  is  likely  to  appear  as  “better  learning.”  The  study  mentions  that  performances 
of  procedures  are  poorly  measured  by  assessments  measuring  some  verbal  skill. 

The  area  of  math  involves  measures  of  visually  oriented  manipulation  as  well  as  application  of 
learned  procedures.  Because  it  contains  both  a need  to  visually  focus  on  a document  and  a sequential 
procedure,  it  is  difficult  to  anticipate  whether  any  failures  in  learning  in  this  area  can  be  avoided  by 
narrating  rather  than  asking  students  to  read  the  procedural  instructions  involved.  It  is  certainly  possible 
that  listening  to  the  procedure  described  while  it  is  being  done  may  be  more  likely  to  lead  to  accurate 
problem  solving  than  reading  the  procedure  from  a textbook.  On  the  other  hand,  the  large  number  of 
visually  based  symbols  required  may  make  a case  for  reading  being  the  better  choice  of  learning  medium. 
This  is  another  area  where  comparison  studies  may  prove  useful. 

The  study’s  result  was  obtained  only  in  a low-reading  adult  population.  But  one  important  detail 
that  must  be  considered  is  that  children,  up  until  approximately  the  sixth  grade,  are  on  the  same  track  as 
low-reading  adults.  The  two  groups  share  a characteristic  of  being  able  to  acquire  information  more  quickly 
from  listening  than  from  reading.  Research  needs  to  be  done  about  the  benefits  of  narrated  presentation  for 
learning  of  skills  other  than  reading.  The  study  at  hand  provided  one  answer  about  the  value  of  narration 
over  text  and  opens  the  door  to  many  new  questions. 

Conclusion: 

The  value  of  auditory  presentation  is  not  as  much  that  students  learn  “better”  but  that  failure  to 
learn  at  all  is  diminished.  This  was  shown  only  for  low  reading  adult  learners  using  a computer  to  learn  a 
computer-based  procedure.  The  possibility  exists  that  the  result  could  extend  to  other  environments  and 
other  populations,  with  learning  of  something  other  than  procedures.  More  research  into  the  value  of 
narration  in  areas  not  touched  by  this  study  needs  to  be  done. 
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Technology  Tools  for  the  Middle  School  Classroom 
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Abstract 

The  purpose  of  this  study  is  to  examine  the  extent  on  how  middle  school  teachers  use  the 
technology  in  the  classroom  and  to  what  degree  their  students  are  using  technology  and  the  facilities, 
equipment  provided  by  their  schools.  To  answer  many  questions  surrounding  technology  in  schools,  a study 
was  conducted  with  randomly  selected  one  thousand  schools  in  ten  states  and  the  data  was  collected  at  the 
end  of  April  2004. 


Introduction 

Technology  continues  to  touch  the  lives  of  all  Americans.  From  computers  and  cell  phones  to 
video  games  and  on-line  bill  paying,  technology  is  everywhere  in  American  society.  In  education, 
especially,  uses  of  technology  in  schools  have  exploded  in  recent  years.  For  example,  the  number  of 
computers  in  schools  continues  to  rise  with  more  than  six  million  in  our  nations  elementary  and  secondary 
schools  (Fulton,  1999).  Also,  a recent  survey  of  the  National  Center  for  Educational  Statistics  has  reported 
99%  of  full-time  public  teachers  have  access  to  computers  or  to  the  Internet  somewhere  in  their  school 
(Roward,  2002).  In  another  major  report  on  technology  and  readiness  in  schools,  it  was  concluded  that  all 
students  should  graduate  with  the  technology  skills  needed  for  today’s  world  and  all  teachers  should  be 
equipped  to  use  technology  as  a tool  to  achieve  high  academic  standards  in  their  classrooms  (CEO  Forum, 
1999).  In  addition,  integrating  technology  in  the  classroom  has  been  proven  to  have  positive  effects  on 
student  learning.  In  their  longitudinal  study  on  the  effects  of  integrating  computer  based  leaning  centers  in 
elementary  schools,  Kromhout  & Buzin  (1993)  pointed  out  that  the  use  of  technology  in  the  classroom 
resulted  in  positive  and  statistically  significantly  high  test  scores,  across  grades  and  schools,  for  the  areas  of 
reading  and  mathematics.  In  a similar  study  by  Lemke,  Quinn,  Zucker,  and  Cahill  (1998),  it  was  reported 
that  students  who  used  computers  in  the  classroom  did  significantly  better  on  standardized  tests  than 
students  who  used  computers  in  a lab  and  that  teachers  were  more  likely  to  integrate  computers  into  their 
instructional  plans  when  the  computers  were  in  their  classrooms. 

In  spite  of  the  rapid  growth  of  technology  in  our  nation's  schools,  problems  have  arisen.  First, 
there  is  evidence  that  many  teachers  still  do  not  use  technology  in  their  classrooms  (Education  Week, 

1999).  Second,  according  to  a survey  (2002)  of  teachers  conducted  by  Center  for  Educational  Statistics, 
only  22%  felt  well  prepared  to  use  computers  in  their  classrooms  (Essex,  2002).  Third,  according  to 
Mouza  (2003)  quality  professional  development  in  technology  for  teachers  has  been  lacking  because  school 
districts  have  been  using  funds  to  purchase  equipment  rather  than  train  their  teachers.  There  are  also  other 
questions  regarding  technology  in  schools.  Are  teachers  adequately  prepared  to  use  technology  in  their 
teaching?  Are  schools  providing  enough  technology  tools  for  their  teachers  and  students?  Do  teachers 
really  value  technology  in  their  teaching?  In  spite  of  these  concerns,  most  educators  continue  to  believe  that 
technology  is  essential  for  quality  education  in  the  United  States.  Middle  schools,  in  particular,  can  play  an 
important  role  in  helping  middle  schoolers  learn  more  effectively,  and  in  improving  their  technology  skills. 

To  answer  some  of  the  questions  surrounding  technology  use  in  middle  schools,  a survey  was  sent 
to  1000  middle  schools  in  eleven  states  (New  Hampshire,  Vermont,  West  Virginia,  Kentucky,  Oklahoma, 
Texas,  South  Dakota,  Massachusetts,  Montana,  Nevada,  and  Iowa).  This  study  explored  the  types  of 
technology  used  by  middle  schools  teachers,  their  value,  and  the  training  and  equipment  provided  them  by 
their  schools.  One  thousand  questionnaires  were  sent  to  these  teachers  and  three  hundred  sixty-three  were 
returned. 

The  questionnaire  consisted  of  two  parts.  In  part  one  the  teachers  marked  how  often  they  used 
particular  types  of  technology  (never,  rarely,  sometimes  and  often)  and  how  valuable  these  types  of 
technology  were  in  their  teaching  (very  valuable,  valuable,  no  valuable,  no  opinion,  little  value  and  no 
value).  In  part  two  the  teachers  indicated  the  availability  of  different  types  of  technology  in  their  schools 
and  the  training  provided  for  them  and  their  students  by  their  schools.  The  results  were  analyzed  using 
percentages  of  uses  and  of  value.  The  majority  of  those  responding  were  female  with  one  to  ten  years  of 
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experience.  Moreover,  most  of  the  respondents  were  mathematics  and  science  teachers  followed  by  those 
teaching  social  studies  and  English.  This  article  will  report  only  those  findings  dealing  with  the  uses  of 
technology  tools  in  middle  schools. 


Results 

The  results  of  this  study  indicate  several  important  findings  concerning  technology  in  middle 
schools.  Table  A shows  that  the  most  often  used  types  of  technology  were:  word  processing  for  teaching 
(83.9%),  computers  in  preparing  lessons  (73.3%),  e-mail  for  discussing  teaching  with  other  educators 
(68.6%),  and  the  overhead  projector  (57.9%).  The  next  most  often  used  forms  of  technology  were:  the 
Internet  (53.9%),  computer  assisted  instruction  (46%),  computer  graphics  (40.7%),  and  Microsoft 
PowerPoint  (40.0%).  Table  A also  reveals  that  there  were  other  types  of  technology  used  less  often  by  the 
teachers:  VCR's  (33.7%),  spreadsheets  for  assisting  teaching  (30.5%),  CD-ROM  disks  (30%),  television 
(26.5%),  web  pages  (25.9%),  video  projectors  (LCD  projectors)  (28.1%),  databases  to  assist  teaching 
(27.6%),  digital  cameras  (26.4%),  and  films  (12.7%).  Additional  evidence  found  in  Table  A indicates  that 
the  least  often  used  types  of  technology  were:  the  tape  recorder  (6.7%),  digital  camcorder  (7.1%),  laser  disc 
player  (4.1%),  radio  (1.7%),  film  strips  (2.0%),  slide  projector  (1.1%),  and  record  player  (.8%). 


Table  A:  Technology  Used  by  Teachers 


Often 

Sometimes 

Rarely 

Never 

Use  of  word  processing  for  teaching 

83.9% 

13.6% 

1.9% 

0.6 

Use  of  computer  to  prepare  lessons 

73.3% 

21.2% 

3.6% 

1.9% 

Use  of  overhead  projector  in  teaching 

57.9% 

21.8% 

10.7% 

9.6% 

Use  of  Internet  for  teaching 

53.9% 

39.1% 

5.0% 

2.0% 

Using  e-mail  to  discuss  teaching  with  other  educators 

68.6% 

19.8% 

6.6% 

5.0% 

Use  of  computer  assisted  instruction 

46.0% 

40.7% 

10.0% 

3.3% 

Use  of  computer  graphics  for  teaching 

40.7% 

37.1% 

13.5% 

8.7% 

Use  of  VCR  in  Teaching 

33.7% 

47.0% 

16.6% 

2.7% 

Use  of  Microsoft  Power  Point  and  other  computer 
programs  for  teaching 

40.0 

31.9% 

17.2% 

10.9% 

Use  of  spreadsheets  for  assisting  teaching 

30.5% 

38.0% 

23.3% 

8.3% 

Use  of  CD-ROM  in  teaching 

30.0% 

39.4% 

26.6% 

6.0% 

Use  of  television  in  teaching 

26.5% 

45.6% 

20.4% 

7.5% 

Create  web  pages  for  teaching 

25.9% 

20.4% 

12.9% 

40.8% 

Use  of  video  projector  in  teaching 

28.1% 

28.7% 

17.1% 

26.1% 

Use  of  database  to  assist  teaching 

27.6% 

29.3% 

24.0% 

19.1% 

Use  of  digital  camera  in  teaching 

26.4% 

32.1% 

23.1% 

18.4% 

Use  of  films  in  teaching 

12.7% 

32.5% 

24.2% 

30.6% 

Use  of  tape  recorder  in  teaching 

6.7% 

20.8% 

31.1% 

31.4% 

Use  of  digital  camcorder  in  teaching 

7.1% 

23.4% 

28.0% 

41.5% 

Use  of  laser  disc  player  in  teaching 

4.1% 

14.9% 

16.5% 

64.5% 

Use  of  radio  in  teaching 

1.7% 

14.9% 

24.9% 

58.5% 

Use  of  film  strips  in  teaching 

2.0% 

7.0% 

18.8% 

72.2% 

Use  of  slide  projector  in  teaching 

1.1% 

7.7% 

22.4% 

68.8% 

Use  of  record  player  in  teaching 

0.8% 

4.4% 

13.3% 

81.5% 

Results  reported  in  Table  B show  the  different  types  of  technology  required  or  encouraged  by 
teachers  for  their  students.  These  were:  encourage  student  use  of  Internet  for  work  (56%),  require 
computers  for  learning  or  assignments  (49.3%),  require  students  to  use  word  processing  for  assignments 
(37.8%),  require  the  use  of  Internet  for  work  (32.1%),  encourage  power  point  use  by  students  (27.5%), 
encourage  students  to  use  computer  graphics  (29.6%),  encourage  e-mail  use  by  students  (28.2%), 
encourage  use  of  different  types  of  media  in  work  (22.8%),  and  encourage  students  to  use  CD-ROM  for 
work  (18.8%).  Further  results  in  Table  B show  that  other  lesser-used  types  of  technology  recommended  for 
students  were:  spreadsheets  (8.2%),  database  (7.8%),  VCR  (7.5%),  digital  camcorder  (5.4%),  and  laser 
disc  players  (1.4%). 
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Table  B:  Technology  Encouraged  or  Required  for  Students 


Often 

Sometimes 

Rarely 

Never 

Encourage  student  use  of  Internet  for  work 

56.0% 

38.2% 

4.7% 

1.1% 

Require  computers  for  learning  or  assignments 

49.3% 

36.2% 

10.0% 

4.5% 

Require  students  to  use  word  processing 

37.8% 

39.4% 

15.3% 

7.5% 

Require  use  of  Internet  for  work 

32.1% 

46.3% 

13.3% 

8.3% 

Encourage  power  point  use  by  students 

27.5% 

36.4% 

19.3% 

16.8% 

Encourage  students  to  use  computer  graphics 

29.6% 

40.6% 

17.1% 

12.7% 

Encourage  use  of  different  types  of  media  in  work 

22.8% 

37.8% 

26.1% 

13.3% 

Encourage  student  to  use  CD-ROM  for  work 

18.8% 

33.4% 

31.2% 

16.6% 

Encourage  student  to  use  electronic  portfolio 

10.5% 

17.1% 

16.9% 

55.5% 

Encourage  e-Mail  use  by  students 

9.7% 

25.8% 

25.5% 

39% 

Encourage  student  use  of  spreadsheets 

8.2% 

25.9% 

36.1% 

29.8% 

Encourage  student  use  of  database 

7.8% 

24.5% 

28.4% 

39.3% 

Encourage  student  use  of  VCR 

7.5% 

26.9% 

34.3% 

31.3% 

Encourage  student  use  of  digital  camera 

15.2% 

28.7% 

28.1% 

28.0% 

Encourage  student  use  of  digital  camcorder 

5.4% 

18.9% 

29.3% 

46.4% 

Encourage  student  use  of  laser  disc  players 

1.4% 

5.6% 

16.2% 

76.8% 

Findings  in  Table  C indicate  that  schools  are  providing  many  types  of  technology  tools  for  their 
teachers.  VCR's  are  provided  for  a majority  of  teachers  (93.3%)  and  Internet  access  is  available  in  most 
classrooms  (95%).  CD-ROM  equipment  is  present  in  over  eighty  percent  of  the  classrooms  (89.1%)  and 
computers  are  provided  for  students  in  more  than  seventy  percent  of  classrooms  (72.2%).  Digital  cameras 
(73.3%)  and  e-mail  facilities  (86.9%)  are  also  being  found  in  over  85%  of  the  schools.  Video  (LCD) 
projectors  are  available  in  over  60%  of  the  schools  (65.6%),  while  technology  training  for  teachers  (91.9%) 
and  special  technology  instruction  for  students  (91.1%)  are  provided  by  most  schools. 


Table  C:  Technology  Provided  for  Teachers  and  Students 


Yes 

No 

VCR's  provided  in  classroom 

93.3% 

6.7% 

Internet  access  in  classroom 

95.0% 

5.0% 

CD-ROM  equipment  provided  in  classroom 

89.1% 

10.9% 

Computers  provided  in  classroom 

72.2% 

27.8% 

Digital  cameras  provided  in  classroom 

73.3% 

26.7% 

Distance  learning  is  provided 

48.3% 

51.7 

E-Mail  in  classroom 

86.9% 

13.1% 

Video  (LCD)  projectors  provided  in  classroom 

65.6% 

34.4% 

Training  for  teachers  in  uses  of  technology 

91.9% 

8.1% 

Instruction  for  students  in  using  technology 

93.1% 

6.9% 

Conclusions 

Evidence  from  this  study  indicates  several  important  findings.  First,  middle  school  teachers  are 
using  several  types  of  technology  tools  in  their  teaching.  Word  processing  and  computers  seem  to  be  the 
most  popular  with  over  70%  of  the  teachers  reporting  that  they  often  use  them.  E-mail,  the  overhead 
projector  and  the  Internet  are  also  popular  with  more  than  half  of  the  teachers  using  them  often  in  their 
classes.  Other  tools  such  as  computer  assisted  instruction,  computer  graphics,  VCR's,  spreadsheets,  CD- 
ROM,  television,  web  pages,  databases,  and  digital  camera,  are  being  frequently  used  but  by  only  25  to 
34%  of  the  teachers.  The  fact  that  the  teachers  are  employing  several  types  of  technology  tools  in  their 
teaching  is  important  because  it  now  shows  that  technology  means  more  than  just  using  computers  to 
enhance  instruction.  Second,  today’s  middle  school  teachers  are  rarely  employing  some  traditionally  used 
of  technology  tools:  films,  film  strips,  slide  projectors  and  tape  recorders.  Third,  middle  school  teachers  are 
clearly  encouraging  (at  times  requiring)  their  students  to  utilize  technology  tools  in  their  work.  For 
example,  nearly  one  half  of  the  teachers  in  the  study  required  their  students  to  use  computers  for  their 
assignments  and  over  50%  encouraged  them  to  utilize  the  Internet.  Word  processing,  power  point,  and 
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computer  graphics  are  further  being  encouraged  by  22  to  37%  of  the  teachers.  By  encouraging  their 
students  to  utilize  technology  tools,  it  seems  clear  that  middle  school  teachers  are  recognizing  the  value  of 
technology  tools  in  helping  students  learn.  Fourth,  middle  school  teachers  are  receiving  technology 
training,  and  students  are  been  provided  with  instruction  in  using  technology  tools.  In  addition,  both  are 
being  provided  with  a variety  of  technology  tools  by  their  schools.  These  facts  are  important  because  they 
appears  to  refute  prior  research  showing  a lack  of  technology  training  for  teachers  and  insufficient  amount 
of  equipment  being  provided  by  schools. 

Although  the  study  indicates  that  many  middle  school  teachers  are  integrating  technology  tools  in 
to  their  teaching,  encouraging  their  students  to  use  them  and  are  being  provided  with  essential  equipment 
and  training,  it  seems  clear  that  middle  school  teachers  need  to  integrate  other  types  of  technology  tools 
into  their  teaching  instead  of  relying  so  much  on  word  processing  and  the  computer.  Web  pages,  digital 
cameras,  video  projectors,  power  point  presentations,  computer  graphics,  and  videos  are  being  used  but 
infrequently.  Students,  too,  are  being  encouraged  to  use  certain  types  technology  tools  for  their  work  but 
need  to  use  other  tools  (such  as  e-mail)  to  enhance  their  learning. 

In  summary,  the  use  of  technology  tools  in  middle  schools  continues  to  grow  each  year. 
Nevertheless,  more  teachers  need  to  use  more  technology  tools  in  their  everyday  teaching.  Lastly,  more 
and  more  middle  school  teachers  are  understanding  that  integrating  a variety  of  technology  tools  into  their 
teaching  is  essential  for  improving  the  achievement  of  students  and  that  using  many  types  of  technology 
tools  will  help  students  to  develop  technology  skills  needed  for  success  in  later  life. 
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Introduction 

Cognitive  science  and  its  theories  have  been  used  in  designing  learning  environments  and  many 
researchers  have  suggested  that  technology  can  assist  in  the  implementation  of  these  theories  (Spiro  et  al, 
1991;  CTGV,  1993;  Jonassen,  2004).  Video  games  have  become  one  of  the  most  influential  forms  of 
technology  in  the  world.  This  paper  examines  the  potentials  of  video  game  technology  as  a cognitive 
apprenticeship-based  learning  environment  by  exploring  the  principles  of  Collins-Brown’ s (1989)  model  of 
cognitive  apprenticeship  with  current  video  games  and  how  the  principles  are  applied  effectively  using  the 
unique  characteristics  of  video  games. 


Cognitive  Apprenticeship 

Situated  Cognition 

Cognitive  apprenticeship  is  situated  within  the  social  constructivist  paradigm  (Brown  et  al.,  1989; 
Resnick,  1989). 

The  theory  of  situated  cognition  claims  that  every  human  thought  is  adapted  to  the  environment 
(Clancey,  1997).  Moreover,  what  people  perceive,  think,  and  do  develops  in  a fundamentally  social  context 
(Driscoll,  2000).  The  cultural  context,  the  co-constitutive  nature  of  individual-action-environment,  and 
multiple  knowledge  communities  have  all  become  elements  of  situated  cognition  theory.  Wenger  (1998) 
summarized  the  basic  premises  of  situated  cognition  theory:  (1)  We  are  social  beings,  which  is  a central 
aspect  of  learning;  (2)  Knowledge  is  a matter  of  competence  with  respect  to  valued  enterprises,  such  as 
singing  in  tune,  discovering  scientific  facts,  fixing  machines,  and  so  on;  (3)  Knowing  is  a matter  of 
participating  in  the  pursuit  of  such  enterprises,  that  is,  active  engagement  in  the  world. 

Cognitive  Apprenticeship  theory 

Based  on  the  limitation  they  perceived  of  in  today’s  formal  schooling,  Collins  and  his  colleagues 
(1989)  proposed  an  alternative  method  of  instruction  - cognitive  apprenticeship,  a synthesis  of  formal 
schooling  and  traditional  apprenticeship.  The  goal  of  cognitive  apprenticeship  is  to  make  the  thinking 
process  of  a learning  activity  visible  to  both  the  students  and  the  teacher.  The  teacher  employs  the  methods 
of  traditional  apprenticeship  (modeling,  coaching,  scaffolding,  and  fading)  to  effectively  guide  student 
leamingollins,  Brown,  and  Newman  (1989)  and  Collins  (1991)  identified  four  aspects  of  cognitive 
apprenticeship-based  learning:  content,  instructional  methods,  sequencing  of  instruction,  and  sociology. 

Content  refers  to  the  different  types  of  knowledge  required  for  expertise  and  includes  domain 
knowledge  of  concepts,  facts,  and  procedures  and  strategic  knowledge  of  an  expert's  ability  to  make  use  of 
concepts,  facts,  and  procedures  to  solve  problems. 

Instructional  methods  are  the  learning  activities  used  during  instruction  to  help  students  construct, 
use,  manage,  and  acquire  new  knowledge.  They  recommended  seven  instructional  methods:  "modeling," 
"coaching,"  "scaffolding,"  "fading,"  "articulating,"  "reflecting"  and  "exploring." 

Application  of  Cognitive  Apprenticeship 

The  basic  methods  of  cognitive  apprenticeship  have  been  successfully  applied  to  teaching  in  the 
domains  of  reading,  writing,  and  mathematics  (Palincsar  & Brown,  1984;  Scardamalia  & al.  1984; 
Scardamalia  & Bereiter,  1985;  Schoenfeld,  1985).  In  these  domains,  they  emphasize  two  things.  First,  the 
method  is  aimed  primarily  at  teaching  the  processes  that  experts  use  to  handle  complex  tasks.  Where 
conceptual  and  factual  knowledge  are  addressed,  cognitive  apprenticeship  emphasizes  their  use  in  problem 
solving  and  carrying  out  tasks.  Conceptual  and  factual  knowledge  are  exemplified  and  situated  in  the 
contexts  of  their  use.  There  is  a dual  focus  on  expert  processes  and  situated  learning.  Second,  it  focuses  on 
the  learning-through-guided-experience  of  the  cognitive  and  metacognitive,  rather  than  physical,  skills  and 
processes. 
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Most  of  the  studies  in  these  domains  incorporate  the  basic  elements  of  cognitive  apprenticeship, 
using  the  methods  of  modeling,  coaching  and  fading  and  of  encouraging  student  reflection  on  their  own 
learning.  Especially  in  Scardamalia  and  Bereiter’s  ‘Procedural  facilitation  of  writing’  case  (1984,  1985), 
they  applied  tenets  of  cognitive  apprenticeship  by  providing  explicit  procedural  supports  based  on 
contrasting  models  of  novice  and  expert  writing  strategies.  Their  approach  was  designed  to  give  students  a 
grasp  of  the  complex  activities  involved  in  expertise  by  explicating  modeling  of  expert  processes,  gradually 
reducing  support  or  scaffolding  for  students  attempting  to  engage  in  the  processes,  and  providing 
opportunities  for  students  to  reflect  on  their  own  and  others’  efforts. 

Cognitive  apprenticeship  has  been  supported  by  media,  mainly  via  the  computer  and  the  Internet. 

Lesgold  and  his  colleagues  (1992)  developed  ‘Sherlock,’  a computer-coached  practice  environment, 
to  develop  the  troubleshooting  skills  of  Air  Force  electronics  technicians.  The  system  is  based  on  the 
cognitive  apprenticeship  theory  of  modeling,  scaffolding  and  coaching  using  a schematic  diagram  that 
demonstrates  expert  understanding  to  the  trainees. 

Jarvela  (1995)  and  Snyder  et  al.  (2000)  demonstrated  the  effectiveness  of  cognitive  apprenticeship 
as  an  instructional  design  approach  for  teaching  technical  skills  in  an  online  environment  by  incorporating 
the  use  of  discussion  databases,  e-mail  and  multimedia.  They  used  cognitive  apprenticeship  techniques 
such  as  expert  modeling,  coaching,  scaffolds,  articulation,  reflection,  and  exploration. 

Video  Game  Technology 

Video  games  have  become  one  of  the  most  influential  forms  of  entertainment  in  the  world.  Video 
games  are  a form  of  popular  culture  very  similar  to  film  or  television  in  their  impact.  Generally,  video 
games’  characteristics  such  as  competition,  interactivity  and  realism  are  presumed  to  be  the  main  factors 
that  affect  users  by  providing  continuous  and  engaging  visceral  and  behavioral  thrills  (Norman,  2004; 
Squire,  2004).  Educators  have  not  researched  video  games  as  much  as  other  media  (computers,  the  Internet, 
film  and  television,  for  example),  but  as  video  games  have  recently  advanced  both  technologically  and  in 
popularity,  there  has  been  a movement  to  study  the  cognitive  potential  of  games  and  how  human  interaction 
and  the  design  of  learning  environments  can  be  supported  through  game  study  (Squire,  2004). 

Research  on  Video  Games  and  Learning 

The  most  significant  attempt  at  researching  video  games  in  a learning  context  was  done  in  Gee’s 
“What  Video  Games  Have  to  Teach  Us  About  Learning  and  Literacy”  (2003)  in  which  he  outlined  several 
learning  principles  by  analyzing  the  various  strengths  and  patterns  used  in  popular  video  games.  These 
principles  can  be  taken  into  consideration  when  we  design  learning  environments. 

Other  educators  see  video  games  as  powerfully  motivating  digital  environments  and 
study  them  in  order  to  determine  how  motivational  components  of  popular  video  games  might  be  integrated 
into  instructional  design  (Bowman,  1982;  Driskell  & Dwyer,  1984;  Bracey,  1992). 

The  “Games  to  Teach”  Project  (Jenkins,  2002),  a collaborative  effort  between  Microsoft 
and  MIT’s  Comparative  Media  Studies  Program,  conducted  a series  of  elaborate  “thought  experiments,” 
and  developed  several  conceptual  prototypes  exploring  different  models  for  how  games  might  enrich  the 
instruction  of  science,  engineering  and  math  at  the  advanced  placement  high  school  and  early  college 
levels.  Their  studies  suggest  that  games  do  offer  teachers  enormous  resources  they  can  use  to  make  their 
subject  matter  come  alive  for  their  students,  motivate  learning,  offer  rich  and  compelling  problems,  model 
the  scientific  process  and  engineering  contexts,  and  enable  more  sophisticated  assessment  mechanisms. 

Video  Games  as  a Cognitive  Apprenticeship-based  Learning  Environment 

Good  video  games  immerse  users  in  rich  interactive  digital  micro-worlds.  Bowman  (1982) 
suggests  that  educators  could  use  video  games  as  a model  for  improving  learning  environments  by 
providing  clear  goals,  challenging  students,  allowing  for  collaboration,  using  criterion-based  assessments, 
giving  students  more  control  over  the  learning  process  and  incorporating  novelty  into  the  environment. 

Many  other  game  researchers  have  implied  that  video  games  could  be  used  to  create  a cognitive 
apprenticeship-based  learning  environment  (Jenkins,  2002;  Prensky,  2002;  Squire,  2004).  However,  none 
of  them  deeply  studied  how  video  games  can  support  certain  learning  theories. 

The  following  are  specific  implications  of  game  technology  and  how  and  why  it  can  be  used  in 
learning  according  to  the  principles  of  Collins-Brown’ s (1989)  model  of  cognitive  apprenticeship. 
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Content:  Teach  tacit,  heuristic  knowledge  as  well  as  textbook  knowledge. 

Collins  et  al.  (1989)  refer  to  four  kinds  of  knowledge:  domain  knowledge,  heuristic  strategies, 
control  strategies  and  learning  strategies.  Domain  knowledge  is  the  conceptual  and  procedural  knowledge 
typically  found  in  textbooks  and  other  instructional  materials.  Heuristic  strategies  are  ‘tricks  of  the  trade’  or 
‘rules  of  thumb’  in  which  people  use  narrow  solution  paths  to  solve  a problem.  Control  strategies  are  for 
students  to  monitor  and  regulate  their  problem-solving  activity. 

Video  games’  high  interactivity  and  rich  multimedia  elements  are  almost  unlimited  in  presenting 
different  knowledge  or  strategies.  In  video  games,  domain  knowledge  can  be  easily  presented  by  providing 
a tutorial  or  presentation  using  media  such  as  text,  narration,  graphics,  video  and  animation. 

Heuristic,  control  and  learning  strategies  can  be  fostered  through  repeated  problem-solving 
practice  (Squire,  2004).  In  the  virtual  environment  of  video  games,  learners  can  repeat  actions  without  the 
anxiety  of  failure.  Also,  by  taking  part  in  simulations,  learners  can  easily  observe  the  natural  uses  of 
domain  knowledge  and  later  apply  it  in  an  authentic  context. 

Most  video  games  first  present  domain  knowledge  and  require  players  to  apply  this  knowledge  in 
problem-solving  activities.  Domain  knowledge  is  especially  necessary  in  role-playing  and  simulation 
games;  without  it  users  often  cannot  succeed  in  the  game.  By  solving  problems  and  achieving  incremental 
goals,  game  players  also  learn  other  strategies.  For  example,  in  the  PC  game  ‘Monopoly  Tycoon,’  users 
first  go  through  an  interactive  tutorial  and  static  text  to  learn  the  basic  words,  concepts  and  strategies  of 
buying  and  managing  real  estate  properties.  When  the  actual  game  starts,  players  use  the  basic  knowledge 
and  strategies  and  domain  knowledge  from  the  tutorial  in  order  to  experiment  with  more  advanced  and  less 
explicit  strategies  for  completing  objectives;  they  ultimately  learn  complex  and  contextualized  real  estate 
management  skills. 

Situated  Learning:  Teach  knowledge  and  skills  in  contexts  that  reflect  the  way  the  knowledge  will  be  useful 
in  real  life. 

Sociology  is  one  of  the  building  blocks  of  the  cognitive  apprenticeship  model,  in  which,  all 
instruction  is  placed  within  ‘authentic’  contexts  that  mirror  real-life  problem  solving  situations.  The 
learning  environment  should  reproduce  the  technological,  social,  temporal  and  motivational  characteristics 
of  real-world  situations  where  what  is  being  learned  will  be  used  (Collins,  Brown  & Newman,  1989). 

Games  have  a great  potential  to  support  this  principle  by  providing  a realistic  environment  in 
which  learners  can  practice  and  apply  newly  acquired  knowledge  and  problem  solving  abilities.  In  ‘CSI: 
Dark  Motives,’  the  player  is  a new  trainee  at  the  Las  Vegas  crime  lab,  and  must  gather  and  analyze 
evidence,  interview  suspects,  and  work  with  other  lab  members  in  order  to  solve  the  crimes  and  be  hired  by 
the  team.  The  gameplay  takes  place  in  a true-to-life  environment:  the  player  has  a believable  incentive  for 
solving  the  crimes  (employment),  the  various  suspects  and  characters  assert  themselves  in  realistic  ways, 
and  authentic  forensic  tools  must  be  mastered  in  order  to  solve  the  cases. 

Role-playing  and  simulation  games  can  also  give  learners  the  experience  of  contextualized 
learning,  in  particular  when  interaction  and  collaboration  are  integral  success  strategies.  Video  game 
consoles  with  multi-player  online  or  wireless  gaming  capabilities  appear  to  be  an  increasingly  important 
part  of  the  video  game  environment  supporting  the  principle  of  situated  learning  (Squire,  2004).  In  many 
Massively  Multiplayer  Online  Role  Playing  Games  (MMORPGs),  character  class  skill  sets  and  areas  are 
designed  so  that  no  one  player  can  survive  without  bonding  with  others.  Even  in  MUDs  (Multiple  User 
Dungeons),  text-based  online  environments,  users  can  collaborate  in  groups  to  complete  quests  and  solve 
puzzles. 


Modeling  and  explaining:  Show  how  a process  unfolds  and  give  reasons  why  it  happens  that  way. 

In  a video  game,  the  modeling  of  processes  and  the  demonstration  of  expert  performances  can  be 
easily  visualized.  Squire  (2004)  suggests  that  computerized  simulations  or  ‘edutainment’  video  games  can 
be  powerful  tools  for  learning  for  four  reasons:  (1)  learners  can  manipulate  otherwise  unalterable  variables, 
(2)  students  can  view  phenomena  from  new  perspectives,  (3)  learners  can  observe  a system’s  behavior  over 
time  and  (4)  students  can  visualize  a system  in  three  dimensions. 

Most  video  games  contain  features  that  model  and  explain  gameplay  using  visual  aids,  allowing  a 
player  to  observe  a process  and  understand  its  underlying  factors.  In  the  first  battle  of  ‘Final  Fantasy 
Tactics,’  you  do  not  have  control  over  all  the  members  of  your  party.  Rather,  you  only  control  the  main 
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character  and  can  watch  how  your  computer-controlled  allies  move,  attack  and  use  items  in  the  battle.  You 
are  free  to  engage  in  the  battle  or  stay  at  the  rear  of  your  party  and  merely  observe  (you  can  rotate  the 
battleground  in  order  to  view  the  battle  from  four  different  directions).  After  this  battle,  you  gain  control  of 
all  the  members  of  your  party.  The  unique  initiation  to  gameplay  in  ‘Final  Fantasy  Tactics’  allows  the 
player  to  understand  the  methods  and  strategies  necessary  for  successful  future  battles. 

Coaching:  Observe  students  as  they  try  to  complete  tasks  and  provide  hints  and  help  when  needed. 

In  the  learner-centered  environments  of  video  games,  the  player  often  requires  guidance  on  what 
to  do  next.  Gee  (2003)  mentioned  that  good  video  games  deal  with  overt  information  and  guidance  on  one 
hand  and  immersion  in  practice  on  the  other. 

Coaching  in  an  apprenticeship-based  video  game  environment  can  be  accomplished  through  a 
variety  of  forms  of  guidance:  from  programmed  feedback  for  certain  inputs  to  virtual  coaches  as  Gee 
(2003)  described  using  the  example  of  Professor  Von  Croy  (a  character  in  the  ‘Tomb  Raider’  game  series) 
who  guides  Lara,  the  heroine,  over  a variety  of  obstacles.  In  driving  simulation  games  such  as  ‘Crazy  Taxi’ 
or  ‘Grand  Theft  Auto  III’,  a navigating  map  with  marks  indicating  safe  houses  or  mission  locations  is 
another  form  of  a hint.  In  the  ‘Sonic  the  Hedgehog’  games,  memo-style  notices  suggest  what  the  user 
should  do  to  finish  a game  stage.  In  the  ‘Monster  Rancher’  series,  an  in-game  assistant  gives  continuous 
information  on  how  your  virtual-pet  monster  is  feeling,  how  well  it  may  do  in  competitions,  and  what  the 
result  of  certain  training  methods  may  be  (spoiling  may  lead  to  a lazy  monster;  attempting  a difficult  task 
may  frustrate  a monster).  By  following  the  assistant’s  advice,  a player  learns  the  most  appropriate  methods 
for  raising  a healthy  and  victorious  monster. 

In  MMORPGs,  less  experienced  players  often  ask  higher- level  players  advice  about  where  to  find 
a particular  shop  in  a city,  which  weapons  or  spells  are  the  most  effective,  or  how  to  defeat  a certain  enemy. 
‘A  Tale  in  the  Desert,’  however,  makes  coaching  an  integral  element  of  gameplay.  In  this  social  experiment 
of  sorts,  players  start  with  nothing  and  learn  to  gather  materials  and  construct  objects  in  order  to  progress  in 
an  Ancient  Egyptian  society.  Various  skills  can  be  learned  in  the  game,  ranging  from  art  or  architecture  to 
leadership  or  the  human  body.  One  of  the  requirements  of  a player  who  decides  to  train  in  leadership  is  to 
become  a ‘mentor’  for  less  experienced  players.  Mentors  are  ‘on-call,’  and  there  are  mentors  specifically 
devoted  to  speakers  of  languages  besides  English.  A player  can  contact  her  mentor  as  often  or  as  seldom  as 
it  is  necessary,  placing  gameplay  and  learning  directly  in  her  Zone  of  Proximal  Development  (Vygotsky, 
1978). 

Virtual  coaches  or  masters  who  help  players  use  some  functions  of  a game,  hint  at  some  goals, 
tactics,  or  strategies  can  be  easily  integrated  into  games. 

Articulation  and  Reflection:  Have  students  think  about  their  actions  and  give  reasons  for  their  decisions  and 
strategies,  thus  making  their  tacit  knowledge  more  explicit;  Have  students  look  back  over  their  efforts  to 
complete  a task  and  analyze  their  own  performance. 

Cognitive  apprenticeship  also  requires  extended  techniques  to  encourage  the  development  of  self- 
correction and  monitoring  skills.  It  encourages  reflection  by  noticing  differences  between  novice  and  expert 
performance  and  by  using  techniques  such  as  abstracted  replay  (Collins  & Brown,  1998). 

In  video  games,  the  replaying  of  learners’  actions  and  strategies  and  giving  feedback  can  facilitate 
articulation  and  reflection.  Malone  (1981)  also  suggested  that  multiple  goal  structures  and  scoring  can  give 
students  feedback  on  their  progress.  Video  games  can  provide  multiple  sources  or  dimensions  of  feedback 
based  on  users’  performance  in  authentic  contexts.  One  example  of  how  video  games  can  prompt  reflection 
is  through  the  slow-motion  replaying  and  reviewing  of  learners’  actions  like  in  many  martial  arts  fighting 
games  such  as  ‘Tekken’,  ‘Street  Fighter’,  and  ‘Soul  Caliber.’  More  advanced  methods  of  prompting 
reflection  are  evidenced  in  ‘Mario  Kart  64,’  in  which  a player  can  race  around  the  track  against  a ‘ghost’ 
kart  (the  driving  data  of  his  best  lap),  and  ‘ESPN  NFL  2K5,’  in  which  the  computer  can  analyze  every 
aspect  of  a player’ s play  style,  save  this  profile  to  a memory  card,  and  allow  that  person  or  a friend  to 
practice  against  her  play  style  and  learn  her  strengths  and  weaknesses.  These  latter  two  examples 
demonstrate  how  video  games  can  support  prediction,  hypothesizing  and  experimentation,  which  focus  a 
student’s  attention  directly  on  her  own  thought  processes  (Herrington  & Oliver,  1995). 

Articulation  is  seen  less  frequently  in  ‘stand-alone’  video  games,  although  some  games,  like  ‘Myst 
IV:  Revelations’  do  contain  ‘journal’  and  ‘photography’  systems  that  allow  a player  to  focus  more  attention 
on  her  cognitive  processes  while  playing  the  game.  However,  MMORPGs  such  as  ‘World  of  Warcraft’  or 
‘Everquest,’  because  of  their  collaborative  nature,  foster  a player’s  frequent  articulation  of  his  knowledge 
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when  negotiating  with  his  group  mates  about  which  dungeon  to  enter,  the  method  of  attack  or  how  to 
complete  a particular  quest.  A player  often  needs  to  not  only  suggest  a course  of  action  to  the  group,  but 
also  to  be  able  to  give  several  reasons  why  and  how  it  should  take  place,  or  why  it  should  happen  one  way 
and  not  another. 

Exploration:  Encourage  students  to  try  out  different  strategies  and  hypotheses  and  observe  their  effects. 

In  the  real  world,  exploration  costs  money,  time,  and  sometimes  has  risks  in  various  situations.  In 
virtual  environments,  especially  in  simulation  and  role-playing  games,  learners  can  apply  as  many  different 
strategies  and  hypotheses  as  they  want. 

Gee  (2003)  suggested  a multiple  routes  principle  from  this  point  of  view:  there  are  multiple  ways  to  make 
progress  or  move  ahead.  This  allows  learners  to  make  choices,  rely  on  their  own  strengths  and  styles  of 
learning  and  problem  solving  while  also  exploring  alternative  styles  (p.108). 

In  designing  video  games,  we  can  allow  students  to  choose  different  ways  to  solve  problems,  and 
this  design  strategy  is  easily  found  in  many  strategic  simulation  games.  For  example,  in  first-person  World 
War  II  simulations  such  as  ‘Battlefield  1942,’  and  ‘Medal  of  Honor,’  players  can  choose  different  types  of 
soldiers  and  apply  different  strategies  to  defend  or  attack  territories.  A novice  player  might  choose  to  fight 
as  a sniper  and  find  himself  very  ineffective  against  a battalion  of  tanks.  Players  can  experiment  with 
different  troop  types  and  combinations,  vehicles  and  offensive  and  defensive  strategies.  These  games  allow 
a player  countless  opportunities  to  plan,  act,  and  adjust  strategies  in  order  to  become  a more  effective  team 
member.  Also,  in  ‘Star  Craft’,  players  can  use  different  strategies  to  fight  back  against  two  other  enemies; 
exploration  of  strategies  also  takes  place  when  dealing  with  more  and  varied  enemies.  By  observing  the 
effect  of  the  strategies  they  choose,  players  can  learn  different  perspectives. 

Sequence:  Present  instruction  in  an  order  from  simple  to  complex,  with  increasing  diversity,  and  global 
before  local  skills. 

Sequencing  involves  the  staging  of  learning  whereby  tasks  are  presented  in  increasing  complexity 
and  diversity  so  that  students  develop  a broad  understanding  of  the  domain  of  expertise.  One  of  the 
common  characteristic  features  of  video  games  are  their  increasing  complexity  and  diversity  as  students 
build  certain  skills  and  knowledge  of  strategies.  In  games,  multiple  difficulty  levels  can  be  provided  to 
adjust  the  game  difficulty  as  learner’s  individual  skills  develop  (Malone,  1981).  In  addition,  reducing  the 
amount  of  help  from  virtual  guidance  or  coaches  as  the  game  progresses  can  be  easily  included  in  a game’s 
design. 

Several  skill-related  games  employ  this  strategy,  usually  providing  some  rewards  as  a motivational 
factor  as  the  learner  gains  more  expertise.  A good  example  is  the  game  ‘SSX  Tricky’,  the  snow  boarding 
role-playing  simulation  game.  In  the  beginning  of  the  game,  the  character  can  only  snowboard  on  easier 
slopes.  After  viewing  demonstrations  of  new  tricks,  then  practicing  and  mastering  these  techniques,  the 
game  allows  the  player  to  advance  to  more  difficult  slopes.  As  a reward  for  mastering  both  tricks  and 
slopes,  a player  is  allowed  to  customize  her  character  with  more  fashionable  outfits  and  boards. 

Conclusion 

The  cognitive  apprenticeship  theory  and  its  effective  implementation  strategy  have  been  built 
through  an  analysis  of  traditional  apprenticeship  and  the  extensive  body  of  cognitive  science  research. 

As  discussed  above,  video  game  technology  has  a great  deal  of  potential  in  designing  learning 
environments.  Besides  its  great  effect  on  learners’  motivation,  video  games  have  many  features  that  can  be 
strong  advantages,  when  compared  to  other  media,  in  designing  cognitive  apprenticeship-based  learning:  it 
provides  realistic  virtual  worlds;  it  is  effective  in  visualizing  and  modeling  processes  and  strategies;  it 
allows  for  high  levels  of  interaction;  and  it  is  effective  in  giving  feedback  and  opportunities  for  practice. 
Applied  properly,  these  features  can  make  cognitive  apprenticeship-based  environments  more  engaging  for 
both  students  and  teachers. 
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Introduction 

According  to  the  National  Council  of  Teachers  of  Mathematics  [NCTM],  educators’  goals  should 
be  to  “create  a coherent  vision  of  what  it  means  to  be  mathematically  literate  both  in  a world  that  relies  on 
calculators  and  computers  to  carry  out  mathematical  procedures  and  in  a world  where  mathematics  is 
rapidly  growing  and  is  extensively  being  applied  in  diverse  fields”  (1995,  f2).  Emphasizing  the  importance 
of  understanding  and  using  mathematics,  the  NCTM  also  notes  that  “in  this  changing  world,  those  who 
understand  and  can  do  mathematics  will  have  significantly  enhanced  opportunities  and  options  for  shaping 
their  futures.  Mathematical  competence  opens  doors  to  productive  futures.  A lack  of  mathematical 
competence  keeps  those  doors  closed”  (2000,  p.5).  Likewise,  the  National  Association  for  the  Education  of 
Young  Children  (NAEYC)  (2002)  states  that  young  children  need  mathematical  understanding  and  skills 
not  only  in  mathematics  classes  but  also  in  science,  social  studies,  and  other  subjects.  Regarding  this 
changing  world  where  mathematics  has  never  been  more  important  in  our  life,  workplace,  and  the  scientific 
and  technical  community,  the  International  Society  for  Technology  in  Education  [ISTE]  draws  attention  to 
the  potential  of  technology  to  change  education  and  improve  students’  learning  and  suggests  educators 
“design  developmentally  appropriate  learning  opportunities  that  apply  technology-enhanced  instructional 
strategies  to  support  the  learners”  in  order  to  maximize  their  learning  (2000,  p.9).  Today,  a number  of 
different  technologies  are  being  used  in  mathematics’  classrooms.  Computers  and  computer  programs  are 
one  of  them  and  have  become  increasingly  visible  in  early  childhood  education  settings  since  the  end  of  the 
1980’s  (Char,  1990;  Clements,  1999,  2002;  Haugland,  1997). 

Although,  children  today  have  access  to  computers  in  libraries,  day  care  centers,  preschool 
programs,  elementary  schools,  community  centers,  and  after-school  programs,  educators  still  continue  to 
debate  the  potential  benefits  and  possible  harm  resulting  from  computer  use  by  young  children.  The  most 
important  concern  researchers  raise  is  that  computer  use  might  replace  other  activities  that  are  essential  for 
children’s  development  (Barnes  & Hill,  1983;  Brouwer-Janse,  Suri,  Yawitz,  Vries,  Fozard  & Coleman, 
1997;  Healy,  1998,  2004;  The  Alliance  for  Childhood,  2000).  The  scholars  argue  that  children  may  spend 
more  time  using  computers  than,  for  instance,  playing  with  blocks  or  playing  with  other  children.  In  other 
words,  computers  would  take  children  “away  from  the  real  development  tests  of  the  young  child”  (Turner, 
1992,  p.33).  In  addition  to  the  concern  above,  researchers  draw  attention  to  other  potential  negative  results 
of  computer  use:  Computers  may  prevent  children  from  building  the  social  skills  that  are  important  for  their 
overall  growth  and  development;  and  computers  may  cause  health  problems,  such  as  repetitive  stress 
injuries,  eyestrain,  and  obesity  (Attewell,  Suazo-Garcia  & Battle,  2003;  Barnes  & Hill,  1983;  Healy,  1998; 
Scoter,  Ellis  & Railsback,  2001;  Shields  & Behrman,  2000;  Subrahmanyam,  Kraut,  Greenfield  & Gross, 
2000;  The  Alliance  for  Childhood,  2000). 

On  the  other  hand,  other  researchers  have  identified  potential  benefits  of  computer  use  for 
children’s  development  (Clements,  1994,  1999,  2002;  Clements,  Nastasi  & Swaminathan,  1993;  Haugland, 
1992,  2000;  Haugland,  Bailey  & Ruiz,  2002;  Liu,  1996;  NAEYC,  2002).  Appropriate  use  of  computers 
may  advance  children’s  socialization  and  language  development  (Clement,  1994,  1999;  Clements,  Nastasi 
& Swaminathan,  1993;  Haugland  & Wright,  1997;  Liu,  1996;  NAEYC,  1996;  Wright  & Shade,  1994).  For 
instance,  at  school,  home  or  at  childcare  centers,  a child  can  be  asked  to  use  a computer  with  a partner, 
which  provides  opportunities  for  them  to  work,  share,  and  communicate  with  each  other.  In  addition, 
developmentally  appropriate  software  may  encourage  children  to  use  their  imagination  and  solve  problems 
(Haugland  & Wright,  1997;  Johanson,  1997;  NAEYC,  1996).  Moreover,  technological  tools  may  increase 
children’s  motivation  (Guthrie  & Richardson,  1995;  Haugland  & Wright,  1997,  Papert,  1993).  For 
example,  Guthrie  and  Richardson  conclude  that  children  were  intrinsically  motivated  to  use  computers  and 
spent  more  time  at  the  computer  than  they  did  in  other  activities.  According  to  Haugland  and  Wright, 
children  are  more  motivated  while  using  computers  since  the  computer  provides  children  “a  world  they 
want  to  explore,  experiment  and  discover”  (1997,  p.13). 

Children  with  special  needs  may  also  benefit  from  technology  in  terms  of  minimizing  the  effects 
of  disability,  improving  quality  of  life,  and  enhancing  social  participation  (Brett,  1997;  Johanson,  1997; 
McComas,  Pivik  & Laflamme,  1998;  Schery  & O’Connor,  1992).  Children  who  have  physical  disabilities 
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can  use  computers  with  special  input  devices,  such  as  alternative  keyboards,  including  programmable, 
miniature,  chording  and  on-screen  keyboards  (Kincaid,  1999;  Lyman  & Mather,  1998).  In  addition,  as 
software  that  has  auditory  information  can  serve  acoustic  learners,  software  that  includes  visual  information 
can  provide  greater  usability  for  visual  learners  (McComas,  Pivik  & Lafiamme,  1998). 

In  order  to  ensure  the  desirable  effects  of  the  computer  uses  mentioned  above,  children  should  be 
provided  with  technology-enriched  learning  experiences  that  are  developmentally  appropriate.  Haugland 
and  her  associates  stress  the  importance  of  effective  software  selection  and  note  that  developmentally 
appropriate  software  facilitates  children’s  learning,  empowers  them  to  become  active  participants  in  their 
own  learning,  and  sparks  their  curiosity  (Haugland,  Ruiz,  2002;  Haugland,  Bailey  & Ruiz,  2002).  “Market 
researchers  tracking  software  trends  have  identified  that  the  largest  software  growth  recently  has  been  in 
titles  and  companies  serving  the  early  childhood  market”  (NAEYC,  1996,  p.ll).  In  addition,  according  to 
the  Software  Publisher  Association  (SPA)  consumer  market  report  (1996),  of  families  who  have  young 
children  and  their  own  home  computer,  70  percent  have  purchased  educational  software  for  their  children 
(SPA,  1996). 

The  purpose  of  this  study  is  to  determine  whether  computer  programs  that  are  identified  as 
bestsellers  are  developmentally  appropriate  for  the  targeted  age  group  by  investigating  the  critical  attributes 
of  mathematical  software  intended  for  use  by  young  children.  Also,  this  study  may  assist  software 
designers,  teachers,  and  parents  in  designing  and  selecting  developmentally  appropriate  educational 
software.  For  the  purposes  of  this  study,  three  “bestseller”  mathematical  programs  designated  for  children 
between  the  ages  of  4 and  7 were  identified:  Reader  Rabbit’s  Math,  Jump  Start  Numbers,  and  Math 
Mission:  The  Race  to  Spectacle  City  Arcade.  The  selected  programs  were  evaluated  against  the  following 
six  criteria:  the  degree  of  child  control,  clear  instruction,  violence  prevention,  technical  features, 
connections  to  the  real  world,  and  process  orientation. 

The  Degree  of  Child  Control 

Scholars  agree  that  educational  software  should  enable  learners  to  individualize  all  activities  in 
software  (DeLaurentiis,  1993;  Haugland  & Shade,  1990;  Khalifa,  Bloor,  Middelton  & Jones,  2000;  Mandel, 
1997;  Shade,  1999;  Spencer,  1986)  in  order  to  help  them  build  their  self-esteem  (Haugland  & Shade,  1990) 
and  increase  their  engagement  and  motivation  (Campbell  & Chapman,  1967;  Fisher,  Blackwell,  Garcia  & 
Greene,  1975).  Likewise,  Mandel  argues  that  “the  best  interface  is  the  one  that  lets  users  do  what  they  want 
to  do,  when  they  want  and  how  they  want  to  do  it”  (1997,  p.  xi).  In  addition,  researchers  found  that 
children  get  annoyed  if  the  animated  characters  keep  telling  them  what  to  do  instead  of  giving  them  an 
opportunity  to  choose  what  they  want  to  do  (Druin,  Bederson,  Hourcade,  Sherman,  Revelle,  Platner 
&Weng,  2001).  Child  control  is  also  essential  because  there  are  simply  not  enough  people  in  schools  and 
child  care  centers  to  assist  children  one-on-one  when  they  use  software  (Shade,  1999);  thus,  children  should 
be  encouraged  to  discover  a variety  of  ways  to  solve  problems  by  themselves  (Haugland  & Shade,  1990). 

Haugland  and  Shade  (1990)  stress  that  children  be  active  participants  who  initiate  and  determine 
the  series  of  events  instead  of  responding  to  programmed  activities,  and  they  argue  that  children  should 
have  an  opportunity  to  set  the  pace  of  the  software.  For  instance,  there  should  be  a feature  that  enables 
students  to  terminate  animations  and  interrupt  audio  with  mouse  clicks,  since  students  are  not  always 
patient  enough  to  sit  through  lengthy  instructions  and  will  not  be  able  to  absorb  much  of  the  information  at 
any  one  time  (Deubel,  2002;  Hanna,  Risden,  Czerwinski  & Alexander,  1999).  Animations  and  audio  may 
distract  and  reduce  students’  ability  to  concentrate  on  the  subject.  Providing  for  greater  student  control  of 
software  features  may  eliminate  those  factors  that  negatively  affect  students’  learning. 

Not  surprisingly,  while  solving  some  potential  problems,  granting  children  more  control  may 
conversely  cause  problems.  Studies  have  indicated  that  including  too  much  freedom  of  choice  might 
decrease  learning  effectiveness,  especially  for  novice  users  (Large,  1996;  Niemiec,  Sikorski  & Walberg, 
1996).  According  to  Niemiec  and  colleagues  (1996),  very  young  children  may  not  be  able  to  make 
educational  decisions  since  they  do  not  have  adequate  knowledge.  In  addition,  Steinberg  concluded  that 
“studies  of  learner  control  in  the  last  decade  support  earlier  findings  that  students  who  have  little  prior 
knowledge  about  a subject  are  likely  to  perform  poorly  under  learner  control”  (1989,  p.l  17).  Thus, 
software  should  be  designed  to  allow  children  to  control  the  software;  however,  this  freedom  should  not 
affect  users’  learning  negatively.  Niemiec  et.  al.  (1996)  suggest  developers  add  a feature  that  “advises 
students  of  their  progress  and  suggests  courses  of  action  but  leaves  the  choice  to  the  learner”  (p.l 59). 

Large,  Beheshti  and  Breuleux  (1998)  conducted  a study  in  order  to  identify  primary  school 
students’  information  seeking  in  a multimedia  environment.  Three  multimedia  CD-ROMs  were  placed  in 
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two  classrooms  for  the  duration  of  a class  project  and  students  were  asked  to  find  information  about  a 
specific  topic.  In  one  multimedia  environment,  the  opening  page  offers  users  an  opportunity  to  start  a new 
game  or  open  a saved  game.  The  scenario  of  this  game  begin  “. . .a  dangerous  secret  mission  inside  the 
stone  walls  of  Baron  Mortimer’s  castle  in  14th  century  Europe”  (p.355)  and  the  authors  note: 

Not  surprisingly,  all  student  groups,  on  their  first  encounter  with  [the  multimedia  CD- 
ROM],  chose  to  undertake  the  game  rather  than  delve  deeper  into  the  interface  in  order 
to  discover  the  other  search  options.  Some  groups  became  engrossed  in  the  game  and 
spent  more  than  45  minutes  on  it  without  explicitly  identifying  relevant  information  for 
their  project... (1998,  p.355). 

Regarding  the  literature,  fortunately,  all  of  the  reviewed  programs  provide  children  with  an 
opportunity  to  have  control  as  children,  where  they  can  manage  the  flow  and  direction  of  activities. 
Specifically,  on  the  first  screen  of  Reader  Rabbit’ s Math,  children  can  either  play  a game  or  practice 
counting,  adding,  and  subtracting.  As  noted  in  the  result  section,  the  opportunity  to  choose  a game  over 
practice  might  decrease  learning  effectiveness,  especially  for  novice  users  (Large,  1996;  Large,  Beheshti  & 
Breuleux,  1998;  Niemiec,  Sikorski  & Walberg,  1996).  Math  Mission  also  includes  games  for  children  to 
play.  The  difference  between  the  two  programs  is  that  children  who  use  Math  Mission  need  to  first  develop 
and  improve  their  mathematical  knowledge  in  order  to  be  allowed  to  play  the  games.  In  short,  in  order  to 
enhance  learning  effectiveness,  designers  could  provide  children  with  games  as  rewards  for  their  progress. 


Use  of  Language 

Language  provides  a means  for  expressing  ideas  and  asking  questions,  the  categories  and  concepts  for 
thinking,  and  the  links  between  the  past  and  the  future  (Aldrich,  2002;  Das,  1995;  Piaget,  1962),  which 
means  that  the  use  of  language  is  an  important  issue  with  regard  to  making  software  more  readily 
understandable  to  children.  Salisbury  (1990)  emphasizes  the  importance  of  language  by  contending, 

“...  speedy  and  efficient  word-recognition  and  word  understanding  is  a critical  prerequisite  for 
successful  reading...”  (p.  25).  Basically,  we  know  that  most  children  who  are  three  to  five  years  of  age 
are  not  reading  and  writing,  at  least  not  conventionally.  Also,  children  who  are  six  to  eight  years  of  age 
may  read  and  write  independently  but  they  are  not  typically  proficient.  Briefly,  language  should  be 
suitable  for  the  target  age  group.  The  NCTE  (2001)  stresses  that  the  language  level  of  the  software 
should  be  assessed  in  order  to  see  whether  it  is  too  advanced  or  too  technical  for  the  targeted  age 
group.  Also,  Haugland  and  Shade  (1990)  state  that  the  software  should  include  clearly  written 
instructions  and  verbal  directions.  Thus,  it  might  be  said  that  clear  language  and  suitable  vocabulary 
are  essential  for  developers  to  consider  since  they  influence  students’  initial  understanding  as  well  as 
ongoing  learning.  In  addition,  children  need  simple  and  direct  statements  about  what  is  expected  from 
them  in  order  to  exhibit  desired  behaviors  (Greenberg,  1992;  Haugland  & Shade,  1990). 

Designers  of  children’s  software  may  identify  particular  words  related  to  a subject  for  learners  to 
master  so  that  learners  may  have  an  opportunity  to  enhance  their  vocabulary  knowledge  on  a specific 
subject.  Lor  instance,  studies  (Adams,  1990;  Chun  & Plass,  1997;  Stahl  & Lairbanks,  1986;  Woof, 
2001)  showed  that  some  elementary  grade  students  who  were  provided  direct  instruction  in  a specific 
body  of  vocabulary  words  not  only  acquired  the  definitions  but  also  applied  these  words  to  raise  their 
“text  intellectual”  capacity.  Woof  (2001)  states  that  technology  can  provide  opportunities  for  children 
to  encounter  new  words  in  multiple  contexts  by  allowing  them  to  access  text  or  graphics.  Also,  she 
also  draws  attention  to  the  devices  and  notes  “devices  that  allow  users  to  click  on  words  to  hear  them 
pronounced  (and  sometimes  defined),  can  enhance  understandings  about  new  words.  Such  "hot  spots" 
can  be  a powerful  learning  tool,  particularly  when  used  in  combination  with  other  modalities”  (Woof, 
2001,  p.173)  that  allow  children  to  click  on  words  to  hear  them  pronounced  and  notes  that  those 
devices  can  enhance  understanding  about  new  words,  as  they  click  on  words  to  hear  them  pronounced. 

The  programs  reviewed  are  educational  programs  that  have  clear  and  understandable  oral 
instructions  that  children  can  listen  to  repeatedly.  And  yet,  these  programs  lack  written  instructions. 
Children  can  see  the  written  instruction  only  when  they  quit  from  the  programs.  This  may  have  the  effect  of 
decreasing  the  motivation  of  students  who  have  hearing  problems,  or  who  prefer  to  read  rather  than  hearing 
instructions.  Therefore,  in  order  to  serve  more  children,  software  should  include  clear  written  and  oral 
instructions  with  direct  statements  about  what  is  expected  from  children  in  order  to  demonstrate  desired 
behaviors. 
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Absence  of  Violence 

Quality  software  should  provide  a safe  environment  that  engages  the  learner  in  experiences  that 
promote  intellectual,  physical,  social,  spiritual,  emotional,  and  moral  growth  and  development  (Bushman  & 
Anderson,  2002;  Haugland  & Wright,  1997;  Heath,  Bresolin  & Rinaldi,  1989;  Huesmann  & Skoric,  2003; 
NAEYC,  1994).  However,  today  violence  is  often  used  in  software  and  on  web  sites  to  attract  children’s 
attention  and,  unfortunately,  some  children  may  find  violence  entertaining.  According  to  the  NAEYC 
statement  position  (1994),  children  who  are  under  seven  or  eight  years  old  have  great  difficulty 
distinguishing  “fantasy  from  reality  and  their  ability  to  comprehend  nuances  of  behavior,  motivation,  or 
moral  complexity  is  limited”  (p.  19);  thus,  children  may  think  that  violence  is  fun.  In  addition,  the 
presence  of  violence  may  cause  children  to  become  less  sensitive  to  the  pain  and  suffering  of  others;  to 
become  more  fearful  of  the  world  around  them;  and  to  be  more  likely  to  act  in  aggressive  or  harmful  ways 
toward  others  (NAEYC,  1994).  In  order  to  emphasize  the  negative  effects  of  violence,  Haugland  (n.d.) 
provides  the  following  example: 

What  do  we  communicate  to  children  when  bombs  are  used  to  destroy  pictures  rather 
than  an  eraser?  In  one  classroom,  children  quickly  placed  objects  on  the  screen  using  a 
drawing  program  for  the  sole  purpose  of  watching  a bomb  explode  the  objects  on  the 
screen.  Children  bombed  the  screen  over  and  over.  In  software  and  web  sites  it  is  critical 
to  recognize  that  children  never  experience  the  consequences  of  their  violent  behavior. 

They  just  quit  and  start  over  again  as  if  they  had  never  done  the  violent  act.  These 
communicates  to  children  that  violence  is  harmless.  Yet  in  reality  when  something  is 
bombed,  it  is  destroyed.  It  is  not  possible  to  click  a button  and  have  objects  and  or  people 
restored  (Selecting  Developmentally  Appropriate  Software,  ^[20). 

Also,  Haugland  and  Wright  (1997)  stress  that  software  should  not  include  any  objects  that  children 
could  use  to  perform  violent  actions,  such  as  guns,  swords,  bombs  or  poison.  There  are  also  several  studies 
that  sought  to  identify  the  effects  on  young  children  of  playing  an  "aggressive"  versus  "nonaggressive" 
video  game  (Calvert  & Tan,  1994;  Irwin  & Gross,  1995).  Based  on  these  studies,  children  who  played  an 
aggressive  video  game  displayed  more  verbal  and  physical  aggression,  had  a higher  heart  rate,  reported 
more  dizziness  and  nausea  than  did  other  children  who  did  not  play  the  aggressive  video  game.  These 
studies  did  not  measure  children's  willingness  to  hurt  anyone  else;  nonetheless,  the  data  presented  raises 
concerns  about  children's  mental  and  psychological  development  that  designers  should  consider. 

Jump  Start  Numbers  includes  swords  and  skulls  in  one  of  the  games.  Although  they  do  not  attempt 
to  use  the  swords  to  indicate  a violent  act,  the  presence  of  these  icons  may  cause  children  to  begin 
associating  violent  objects  with  fun.  Also,  in  the  program  there  is  a character  called  Alien  Mutt  who  while 
flying  in  her  spaceship  shoots  the  asteroids  that  Frankie  needs  and  children  have  to  knock  her  off  the 
screen.  NAEYC  states  that  young  children  may  not  be  capable  of  distinguishing  reality  from  fantasy. 
Children  who  use  Jump  Start  Numbers  may  think  that  shooting  something  or  knocking  someone  off  is  fun. 
As  mentioned  before,  in  order  to  provide  children  with  experiences  that  promote  their  intellectual,  physical, 
social,  spiritual,  emotional,  and  moral  growth  and  development,  designers  should  create  software  that  is 
free  of  violent  characters  and  actions. 


Technical  Features 

The  process  of  installation  of  software  should  be  easy  for  children  (Boivie,  1998;  Haugland  & 
Wright,  1997).  Directions  for  loading  the  software  must  be  concise,  specific  and  easy  to  follow.  Directions 
should  include  graphics  and/or  verbal  instructions  whenever  necessary  to  make  it  easy  to  follow  each  step 
of  the  installation.  Also,  directions  should  include  a trouble  shooting  guide  for  difficulties  that  may  emerge 
during  installation  and  while  operating  the  system  (Haugland  & Wright,  1997;  Shade,  1999). 

Once  installed,  software  should  run  quickly  throughout  the  activities  (Chafe,  1998;  Cottrell  & 
Eisenberg,  1997;  Haugland  & Shade,  1990;  Haugland  & Wright,  1997;  Khalifa,  Bloor,  Middelton  & Jones, 
2000;  Shade,  1999).  In  other  words,  there  should  not  be  long  pauses  during  processing  time  that  are  due  to 
either  hardware  capabilities  or  the  inclusion  of  too  many  graphics  or  animations  (Chafe,  1998).  If  software 
runs  slowly  children  may  get  confused  and  wonder:  Did  I click  the  right  spot?  Should  I have  clicked  twice 
instead  of  once?  (Haugland  & Wright,  1997,  p.50).  Also  children  may  get  bored  and  start  checking  what  is 
happening  on  the  screen,  walk  over  to  another  activity,  look  around  to  see  what  is  happening  in  other  areas 
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of  the  room  and  wonder  why  another  child  is  using  their  computer  (Haugland  & Shade,  1990;  Haugland  & 
Wright,  1997). 

Another  important  technical  feature  is  “saving”  children’s  progress  within  the  program  (Haugland 
& Wright,  1997;  Khalifa,  Bloor,  Middelton  & Jones,  2000;  NCTE,  2001;  Shade,  1999).  Khalifa,  Bloor, 
Middelton  and  Jones  (2000)  consider  saving  as  an  opportunity  that  makes  children  know  they  can  stop  and 
revisit  the  software  at  any  time  and  that  is  given  to  students  to  reflect  over  time  on  an  activity,  make 
changes  and  enhance  the  learning  experience.  There  are  several  reasons  children  may  be  interrupted  during 
their  computer  experiences.  For  instance,  it  might  be  time  for  lunch  or  dinner,  or  they  may  have  to  leave 
from  the  place  that  they  were  using  the  software  (Haugland  & Wright,  1997;  Shade,  1999).  Also, 
according  to  the  NCTE,  software  should  allow  learners  to  choose  the  place  where  the  saved  file  goes. 
Software  that  provides  a feature  that  enables  children  to  save  what  they  do  might  prevent  children  from 
having  to  start  over  repeatedly. 

In  addition  to  saving,  a capacity  for  printing  is  also  an  important  component  of  the  software’s 
technical  quality.  Khalifa,  Bloor,  Middelton  and  Jones  state  that  a printing  capacity  is  a feature  that  quality 
software  has  and  the  authors  note  that  “printing  gives  children  more  opportunities  to  integrate  their 
computer  experience  with  concrete  activities”  (2000,  p.5).  Children,  parents  or  teachers  can  easily  record 
what  children  have  done  by  printing  so  that  they  may  have  an  opportunity  to  talk  later  with  children  about 
what  they  did  helping  children  to  improve  their  verbal  and  recall  skills  (Haugland  & Wright,  1997). 

Additionally,  “help”  features  that  include  tutorials,  hints,  sample  problems,  and  reference  libraries 
might  assist  students  in  working  independently  with  software  (Boivie,  1998;  Bush,  1992;  Deubel,  2002). 
According  to  Squires  and  Preece  (1999),  “any  information  [in  help  section]  should  be  easy  to  search, 
focused  on  the  user’s  task,  list  concrete  steps  to  be  carried  out. . .”  (p.473),  since  learners  might  become 
easily  disoriented  without  clear  guidance  and  consistency.  Likewise,  Bush  (1992)  suggests  that  software 
manuals  should  provide  step-by  step  training  focused  on  a particular  activity. 

The  five  important  technical  features  that  quality  software  should  have  are  listed  above.  The  first 
feature  is  its  installation  process.  Except  for  Reader  Rabbit’s  Math,  the  software’s  opening  screen  appears 
automatically  after  the  program’s  CD  is  inserted  into  the  drive.  Users  can  install  the  programs  by  clicking 
the  “install”  button  on  the  opening  page  and  it  takes  between  25  seconds  and  90  seconds  for  each  program. 
It  is  unlikely  that  users  will  face  a problem  as  long  as  the  software’s  system  requirements  meet  the  capacity 
of  computer’s  system.  Briefly,  designers  should  identify  the  software’s  requirements  clearly  and  explain 
them  to  users  in  the  users’  guide.  The  second  one  is  that,  once  installed,  software  should  run  quickly.  With 
the  exception  of  Reader  Rabbit’s  Math,  all  programs  run  without  any  pauses  between  activities.  Where  long 
pauses  do  exist  in  programs,  children  may  become  bored  or  confused. 

Third,  software  should  have  a feature  that  saves  children’s  work  in  order  to  prevent  them  from 
having  to  start  over  repeatedly.  More  specifically,  software  should  remember  all  users  who  have  used  the 
software  previously,  as  well  as  the  challenge  level  each  user  has  completed.  This  feature  may  increase 
children’s  sense  of  importance  and  stimulate  their  engagement.  The  next  feature  is  its  printing  capacity. 

This  feature  enables  children,  parents,  and  teachers  to  record  children’s  works  and  to  talk  about  them, 
which  may  increase  children’s  verbal  and  recall  skills.  Surprisingly,  none  of  the  reviewed  programs 
includes  a printing  capacity.  The  final  feature  is  the  help  section  that  includes  tutorials,  sample  problems, 
and  reference  libraries. 


Screen  Design 

Characters  or  icons  appearing  on  the  screen  should  be  chosen  carefully  to  help  users  recognize  them 
immediately  so  that  they  can  make  a connection  between  software  and  the  real  world  (Large,  Beheshti 
& Rahman,  2002,  p.  90).  For  instance,  designers  may  use  doors  that  refer  users  to  go  outside , or  exit. 
In  addition  to  the  previous  example,  designers  may  use  green  buttons  that  enable  children  to  continue 
navigation,  and  red  buttons  to  tell  children  to  wait  (Haugland  & Wright,  1997;  Jacso,  2003).  In 
addition  to  these  examples,  developers  may  use  a trash  can  to  signal  that  if  you  want  to  delete 
something  just  drag  and  drop  it  into  the  trash  can.  Furthermore,  it  is  suggested  that  icons  should 
represent  children’s  everyday  world  as  well  as  the  context  of  the  learning  task  (Deubel,  2002;  Large, 
Beheshti  & Rahman,  2002;  Squires  & Preece,  1999).  For  instance,  developers  may  use  “side”  for  the 
length  of  the  square  rather  than  “x”.  Also,  in  order  that  students  know  what  an  icon  means,  software 
designers  may  add  verbal  cues  that  appear  when  the  cursor  is  dragged  across  the  icon  (Houser,  1996). 
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We  know  that  still  and  animated  images  can  enhance  the  presentation  of  information  and  have  a 
great  impact  on  students’  learning  (Darves,  Oviatt  & Coulston,  2002;  Large,  Beheshti,  Breuleux  & Renaud, 
1996;  Levin,  Anglin  & Carney,  1987;  Rieber,  1990;  Spencer,  1986;  Swenson  & Anderson,  1982).  Darves, 
Oviatt  and  Coulston  (2002)  carried  out  a study  to  explore  whether  conversational  interaction  with  animated 
characters  may  influence  learners’  motivation,  students’  learning-oriented  behavior  and  the  quantity  of 
impulsive  question  asking.  The  authors  worked  with  twenty  four  elementary  school  children  using  a 
program  called  “I  See!”  that  taught  children  about  24  different  marine  animals.  In  the  wizard,  the  marine 
animals  were  animated  and  could  answer  questions  about  themselves  using  text-to- speech  (TTS)  output. 
Based  on  the  outcome  of  this  study,  Darves,  Oviatt  and  Coulston  concluded  that  animated  characters  with 
extroverted  voice  have  a positive  influence  on  children’s  motivation.  Also,  they  found  that  the  interface 
was  highly  engaging  for  children  and  they  were  excited  about  talking  to  animals.  However,  the  authors 
point  out  that  the  voice  the  animated  character  used  should  be  matched  to  the  character  in  order  to  ensure  a 
desirable  impact  on  users’  performance. 

In  contrast,  we  now  recognize  that  still  or  animated  images  can  also  have  a deleterious  effect. 
Khalifa,  Bloor,  Middelton  and  Jones  draw  attention  to  the  response  that  follows  the  learner’s  answer. 
Khalifa  and  colleagues  state  that  reinforcements  given  for  correct  answer  should  be  more  attractive  than 
that  given  for  incorrect  answer;  otherwise,  learners  “purposely  answer  incorrectly,  hoping  to  see  an 
attractive  display  that  would  not  see  for  correct  answers”  (2000,  p.4).  As  supportive  of  Khalifa  and 
colleagues,  Swenson  and  Anderson  (1982)  provide  an  example  that  emphasizes  the  importance  of  the 
potential  for  unintentional  negative  reinforcement.  The  authors  note  that 

. . .the  student  made  an  incorrect  response,  Snoopy  made  his  appearance  by  walking 
upside  down  across  the  top  of  the  video  screen,  dropping  his  sign  about  halfway  across. 

Not  too  surprisingly,  many  students  deliberately  choose  wrong  answers  so  they  could  see 
Snoopy  upside  down  on  the  screen,  which  was  much  more  interesting  than  the  intended 
reinforcer  (p.134). 

Rusthon  and  Larkin  (2001)  have  reviewed  the  literature  about  connecting  developmentally 
appropriate  practices  to  brain  research.  They  found  that  there  is  a need  for  educators  to  provide  more 
opportunity  for  children  to  use  “music,  bodily-kinesthetic,  visual- spatial,  and  interpersonal  domains  to  learn 
and  express  understanding”  (p.31).  Also,  they  note  that  music,  rhyme  and  rhythm  are  directly  connected 
with  the  other  basic  functions  of  the  brain,  such  as  emotion,  perception,  memory  and  even  language.  In 
addition  to  music,  rhyme  and  rhythm,  Rusthon  and  Larkin  state  that  pictures,  symbols,  and  strong  simple 
images  have  powerful  influences  on  students’  learning.  In  other  words,  pictures,  symbols  and  images  can 
help  children  keep  information  in  memory  for  a long  time.  In  brief,  they  concluded  that  art  should  hold  an 
important  place  in  the  curriculum.  In  addition,  colorful  graphics  and  interesting  animations  keep  children 
more  focused  on  the  subject.  Wileman  (1993)  stresses  the  importance  of  graphics  and  stills  and  says 
children  may  encode  and  keep  content  in  memory  longer  as  well  as  retrieve  it  more  easily  through  the  use 
of  well-designed  visuals.  Researchers  found  that  graphics  and  animation  improve  students’  success 
(Caputo,  1981;  Eiser,  1988;  Forgan  & Weber,  2001;  Hammond,  1971;  Singleton,  2001). 

In  conclusion,  all  characters  in  the  programs  are  easy  for  children  to  recognize  since  their 
appearances  are  similar  to  what  can  be  found  in  their  daily  life.  For  instance,  in  Reader  Rabbit’s  Math,  the 
size  of  animals  corresponds  to  their  size  in  the  real  world.  For  instance,  a hippo  appears  much  smaller  than 
an  elephant.  In  addition  to  the  previous  example,  Math  Mission  teaches  children  about  a cent,  a nickel,  a 
dime,  and  a quarter  and  each  one  is  sized  correctly.  However,  some  icons  in  the  programs  are  not 
meaningful  for  children.  For  instance,  in  order  to  quit  from  the  program,  children  need  to  click  on  a clock 
or  a red  oval  button  with  the  word  “EXIT”  on  it.  The  clock  and  the  button  do  not  have  a verbal  or  oral  cue 
appearing  when  the  curser  is  dragged  across  them.  Children  may  not  be  aware  that  those  icons  will  allow 
them  to  quit  from  the  program  if  they  click  them.  Instead  of  using  a clock  or  a button,  designers  may  use 
doors  to  indicate  that  children  can  quit  the  software  (Haugland  & Wright,  1997;  Jacso,  2003).  In  addition, 
as  noted  above,  software  should  have  colorful,  realistic  graphics,  and  sound  effects  to  improve  students’ 
learning  opportunity  in  terms  of  helping  children  keep  information  in  their  memory  for  a long  time.  Math 
Mission  and  Jump  Start  Numbers  have  attractive  and  colorful  animations.  For  instance,  in  Math  Mission, 
characters  dance,  walk  in  the  city,  deliver  the  daily  newspaper  or  drive  a bus.  Although  Reader  Rabbit’s 
Math  does  have  colorful  graphics,  children  can  easily  anticipate  what  will  happen  next.  This  may  cause 
children  to  experience  a decrease  or  even  a loss  in  motivation.  Thus,  while  it  might  be  concluded  that 
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software  should  include  colorful  graphics,  sounds  and  interesting  animations,  they  should  also  be 
supportive  of  children’s  curiosity  to  discover. 


Process  Orientation 

It  is  important  for  quality  software  to  enable  children  to  persist  at  a task  for  a longer  time,  to  be 
eager  to  learn,  and  to  pay  more  attention  to  the  subject  matter  (Bransford,  Brown  & Cocking,  2000; 
Haugland  & Wright,  1997;  Haugland  & Shade,  1990;  Kinzie,  Sullivan  & Burdel,  1992;  Papert,  1980; 
Seymour,  Sullivan,  Story  & Mosley,  1987;  Singleton,  2001;  Swenson  & Anderson,  1982).  In  Reader 
Rabbit’s  Math,  animations  repeat  themselves,  and  this  may  cause  children  to  become  bored  and  thus 
decrease  their  motivation  to  continue  with  the  software.  At  the  beginning  of  Jump  Start  Numbers,  children 
may  persist  at  collecting  biscuits  for  Frankie  since  they  do  not  know  what  will  happen  if  they  collect  all  the 
biscuits  and  then  wake  Frankie  up.  However,  the  software  does  not  include  the  icon  that  enables  children  to 
print  out  their  certificate  that  is  given  as  a children’s  reward  after  they  find  or  earn  all  the  biscuits.  This  may 
cause  a decrease  in  motivation  as  well  as  the  development  of  distrust  in  the  software. 

On  the  other  hand,  Math  Mission  includes  all  the  icons  that  are  mentioned  in  the  user’s  guide. 
Children  earn  money  as  they  help  each  clerk  in  each  store.  However,  they  are  not  allowed  to  spend  this 
money  until  they  help  all  the  clerks  in  the  Spectacle  City.  After  children  help  all  clerks  and  earn  enough 
money,  they  are  able  to  play  games.  Also,  as  children  play  the  games,  they  have  to  pay  money  for  each 
game  so  that  if  they  spend  all  the  money  they  have,  then  they  need  to  help  more  people.  This  may  increase 
their  eagerness  to  help  more  people  to  earn  more  money.  In  other  words,  the  software  provides  children 
with  games  that  serve  to  positively  reinforce  their  progress.  Therefore,  software  should  have  an  appropriate 
flow  that  keeps  children’ s motivation  level  high.  Returning  to  a prior  recommendation,  designers  may 
provide  children  games  with  different  scenarios,  increasing  children’s  curiosity  and  enabling  them  to 
discover  and  learn 

Reader  Rabbit’s  Math  provides  two  options  for  children  on  the  beginning  page:  The  first  option 
enables  children  to  practice  numbers,  addition,  and  subtraction  in  the  circus.  Surprisingly,  in  this  section, 
children  are  just  expected  to  respond  to  questions  asked  by  the  rabbit.  The  second  option  allows  children  to 
play  three  different  games.  One  game  called  the  Log  Ride  asks  children  to  build  a ladder  to  help  the  rabbit 
reach  his  boat.  Depending  on  the  selected  level,  the  ladder  should  be  10,  20,  30  or  40  rungs  tall.  Children 
are  provided  several  ladder  pieces  that  have  different  heights.  For  instance,  children  can  use  two  5-rung- 
tall  ladder  pieces  or  one  5-rung-tall,  one  2-rung-tall  and  one  3-rung-tall  ladder  to  make  a 10  rung-tall 
ladder.  In  short,  this  may  increase  children’s  curiosity  about  discovering  new  ways  to  build  a right  height 
ladder  so  that  they  may  spend  longer  time  on  playing  this  game. 

Conclusion 

As  noted  in  the  introductory,  although  computers  and  computer  programs  are  one  of  the 
technologies  that  are  being  used  in  mathematics  classrooms,  the  most  important  challenge  is  to  provide 
children  developmentally  appropriate  programs  that  meet  children’s  unique  needs.  In  order  to  assist 
designers  in  designing  developmentally  appropriate  software,  the  focus  of  this  study  was  based  on  an 
investigation  of  critical  attributes  of  mathematical  software  intended  for  use  by  young  children.  In  addition, 
parents  and  teachers  may  find  this  study  beneficial  as  they  purchase  software  for  their  children  or  students. 

Although  this  study  was  effective  in  evaluating  the  “bestseller”  educational  programs  to  determine 
whether  they  are  developmentally  appropriate  or  not  for  the  targeted  age  group,  there  are  two  important 
limitations  of  this  study  to  consider.  First,  during  the  design  process  as  well  as  evaluation  process, 
designers  and  educators  should  listen,  observe  and  work  with  children  in  order  to  address  children’ s unique 
needs.  However,  in  this  study  children’s  preferences  were  considered  based  only  on  information  reviewed 
in  the  literature,  not  from  actual  observations  of  children  using  the  software.  For  future  research,  it  might  be 
suggested  that  designers  and  investigators  consider  children  as  learners,  critics,  inventors  and  technology 
design  partners  during  the  process  of  educational  software  design.  Second,  although  it  has  been  identified 
that  the  largest  software  growth  recently  has  been  in  titles  and  companies  serving  the  early  childhood 
market  (NAEYC,  1996),  this  study  evaluated  only  three  educational  programs.  For  future  research, 
investigators  may  evaluate  more  educational  programs  in  order  to  better  inform  designers  in  their 
development  of  developmentally  appropriate  software. 
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Abstract 

A trend  among  colleges  and  universities  is  to  centralize  academic  advisement  programs  which 
provide  academic  counseling  to  students  at  risk  and  help  them  he  successful  in  an  increasingly  competitive 
environment.  A systemic  analysis  of  performance  was  applied  to  the  program  using  ADDIE  instructional 
design  mode,  the  Goals  Grid  Model  ( Nichols , 2003),  and  Human  Performance  Engineering  Model( Gilbert, 
1996).  The  performance  analysis  looked  into  five  aspects  of  (actuals,  optimals,  feelings,  causes,  and 
solutions)  (Rossett,  1987)  the  performance  by  individual  academic  advisers  to  determine  the  program 
performance  gaps  and  to  recommend  interventions  to  improve  its  performance. 

Introduction 

Long  held  as  a valuable  part  of  the  educational  process  in  higher  education,  academic  advisement 
has  moved  away  from  college  departments  into  centralized  advisement  centers.  By  transferring  academic 
advisement  out  of  the  hands  of  faculty  to  dedicated  staff  in  academic  services  divisions,  universities  have 
realized  benefits  such  as  increased  graduation  rates. 

In  a recent  news  article  about  the  six-year  graduation  rates  among  one  particular  southern  state,  a 
sharp  discrepancy  between  universities  was  revealed.  Many  variables  affect  the  six-year  graduation  rate, 
but  academic  advisement  is  considered  by  many  to  be  a dependant  variable  for  which  universities  have 
considerable  influence.  The  assumption  has  been  that  if  appropriate  academic  advisement  programs  were 
catered  to  the  needs  of  students  then  more  students  would  graduate. 

A systemic  analysis  of  performance  was  applied  to  the  program  using  ADDIE  instructional  design 
mode,  the  Goals  Grid  Model  (Nichols,  2004),  and  Human  Performance  Engineering  Model  (Gilbert,  1996). 
The  performance  analysis  looked  into  five  aspects  of  (actuals,  optimals,  feelings,  causes,  and  solutions) 
(Rossett,  1998)  the  performance  by  individual  academic  advisers  to  determine  the  program  performance 
gaps  and  to  recommend  interventions  to  improve  its  performance. 

Human  Performance  Technology  Approach  towards  Performance  Problems 

There  have  been  many  attempts  to  define  human  performance  technology  (HPT).  However,  no 
single  definition  is  likely  to  obtain  consensus  from  HPT  practitioners  and  researchers  worldwide 
(Stolovitch  & Keeps,  1999).  For  this  study  HPT  was  defined  as  a set  of  systemic  methods  and  processes 
for  solving  problems  related  to  the  performance  of  people  in  organizations.  HPT  looks  at  a specific 
performer  and  zooms  outward  in  scope  to  see  how  that  performer  is  doing  in  relation  to  the  values  of  an 
organization.  In  the  case  study  presented  here,  HPT  approach  is  applied  to  an  academic  advisement 
program  and  specifically  the  academic  advisers  themselves.  The  intent  is  to  apply  systematic  analysis 
techniques  to  explain  to  what  extent  the  advisement  program  is  successful  in  fulfilling  its  promise  to 
students  and  how  the  performance  of  individual  academic  advisor  can  be  enhanced. 

Mission  of  the  Organization 

What  was  once  the  sole  responsibility  of  faculty;  academic  advising  is  now  the  responsibility  of 
centralized  academic  advisement  programs  in  many  colleges  and  universities.  The  department  in  this 
project,  The  University  Advisement  Center  (UAC)  is  a centralized  academic  advisement  center  at  a large 
state  university.  The  mission  of  the  UAC  is  to  provide  academic  advisement  and  developmental  counseling 
to  entering  freshmen,  students  with  less  than  42  hours  of  transfer  credit,  students  who  are  undecided  on 
major  and  at-risk  students. 
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Target  Performers 

The  UAC  is  a department  of  nine  academic  advisers.  Each  adviser  is  responsible  for  providing 
developmental  counseling  to  a portion  of  approximately  10,000  total  students.  With  backgrounds  in  a 
diverse  array  of  academic  disciplines,  all  academic  advisers  share  in  common  successful  completion  of  a 
college  degree.  Many  advisers  hold  graduate  degrees  and/or  have  greater  than  five  years  of  experience 
providing  developmental  counseling  in  higher  education.  Emphasis  on  quality  student  service  is  a value 
that  permeates  the  UAC  culture.  As  the  pay  scale  is  relatively  low  compared  to  other  jobs  in  higher 
education,  advisers  gravitate  toward  their  roles  and  responsibilities  for  the  intrinsic  value  of  providing  help 
to  others.  Academic  advisers  must  be  able  to  be  empathetic,  listen  attentively,  identify  student  concerns, 
potential  problems,  and  develop  custom  strategies  that  utilize  multiple  resources. 

Performance  Problems 

At  the  institution  used  in  this  study,  at-risk  students  have  four  chances  before  becoming 
permanently  excluded  from  the  university.  The  program  described  here  is  an  intrusive  advisement  program 
for  students  who  have  fallen  from  academic  standing  of  good  to  either  supervision  or  probation.  Academic 
good  standing  is  defined  at  the  university  used  in  this  paper  by  a GPA  of  a 2.0  on  a 4.0-scale.  Once  a 
student’s  GPA  drops  below  a 2.0  their  academic  standing  falls  to  warning.  At  this  point  the  student  must 
maintain  a term-GPA  of  a 2.3  while  taking  no  less  then  6-credit  hours.  If  they  neglect  to  meet  these  criteria, 
then  they  fall  to  academic  supervision. 

Once  at  supervision,  students  are  required  to  seek  academic  advisement  before  registering  for 
future  terms.  The  same  criteria  for  warning  are  applied  to  supervision  and  if  at  any  time  a student’s 
cumulative  GPA  rises  to  a 2.0  then  they  return  to  good  standing.  On  the  other  hand,  if  they  neglect  to  meet 
the  requirements  for  supervision,  students  fall  to  academic  probation,  which  is  the  final  standing  before 
permanent  exclusion.  Through  the  process  of  academic  advisement,  these  at-risk  students  are  promised 
academic  assistance.  Assuming  that  advisement  is  effective,  one  might  expect  that  a very  low  rate  of 
students  in  this  program  actually  fall  to  academic  exclusion. 

Performance  Environment 

Academic  advisers  meet  one-on-one  with  students  in  a private  office  setting.  Virtually  all 
counseling  is  conducted  in  person.  A limited  amount  of  information  between  the  adviser  and  student  may 
be  conducted  by  email  or  phone.  Resources  that  the  adviser  may  use  include  student  records,  academic 
evaluations  that  include  the  degree  requirements  for  a student’s  major,  the  undergraduate  catalog,  other 
university  resources  such  as  the  Career  Center,  or  Internet  resources. 

Target  Performance 

Optimal  performance  for  an  academic  adviser  should  result  in  a low  rate  of  students  falling  to 
exclusion.  Although  historical  trends  for  this  particular  program  show  a decrease  in  the  number  of  students 
who  fall  to  exclusion,  many  questions  remain  about  its  actual  effectiveness.  Possible  reasons  for  a 
discrepancy  among  advisers  in  numbers  of  students  who  fall  to  exclusion  could  be  due  to  variance  in 
developmental  counseling  approaches,  time  allotted  for  individual  advisement  sessions,  resources  utilized, 
or  a myriad  of  other  issues.  An  issue  this  analysis  project  hoped  to  shed  light  upon  was  defining  measurable 
outcomes  for  academic  advisers  in  their  work  with  at-risk  students. 

Many  academic  advisers  feel  confident  about  their  ability  to  help  students  improve  their  GPA. 
However,  when  period  reports  are  released  with  numbers  of  students  who  fall  to  exclusion,  many  advisers 
feel  a sense  of  loss  about  the  students  who  have  failed  to  succeed.  The  significance  of  improving  the 
performance  gap  could  result  in  a two-fold  effect.  First,  advisers  would  feel  a sense  of  accomplishment. 
Second,  the  university  would  benefit  by  retaining  more  students. 

Guiding  Models  Applied  in  the  Project 

A number  of  models  were  used  in  this  project  of  performance  analysis:  ADDIE  instructional 
design  model,  the  Goals  Grid  Model  (Nichols,  2004),  the  Human  Performance  Engineering  Model  (Gilbert, 
1996)  and  Rossett’s  needs  assessment  model  (of  actuals,  optimals,  feelings,  causes,  and  solutions)  (1998). 

ADDIE  Instructional  Design  Model 
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ADDIE  model  was  used  throughout  the  performance  analysis  that  guided  us  as  a management 
process.  Key  questions  and  tasks  in  the  project  were  listed  in  Table  1. 

Project  Phases 

Tablet:  ADDIE  Process  in  the  Project 

Key  Questions  and  Tasks 

Analysis  Phase 

What  does  exemplary  performance  look  like? 
Identifying  the  exemplar/  Calculating  PIP 
How  will  we  measure  the  performance? 

Design  Phase 

What  analysis  tools  are  most  appropriate  to  conduct  the 
assessment? 

Tool  Selection/  Identify  Stake  Holders  & Target  Audience 

Development  Phase 

How  do  we  adapt  our  tools  to  best  meet  the  needs 
assessment? 

Develop  Tools/  Identify  Resources 

Implementation  Phase 

What  can  the  performers  tell  us  about  the  performance? 
Gather  Data 

Evaluation  Phase 

Formative  Evaluation:  Are  we  asking  the  right  questions? 
Revise  Project  Statement 
What  interventions  can  improve  the  performance? 
Recommend  Interventions 

The  Goals  Grid  Model 

The  Goals  Grid  Model  (Nichols,  2003)  was  used  to  develop  the  project  statement.  It  helped  to 
establish  a clear  goals  and  objectives  of  our  project. 

Table  2:  The  Goals  Grid  Model  in  Establish  Project  Statement 

Goals  Grid  Model 


Questions Tasks  to  Clarify  the  Project  Goals 


What  are  you  trying  to 
achieve ? What  you 
want  but  don’t  have. 

Identified  by  the  standards  of  exemplary  performance  and 
repertoire  of  skills  necessary  for  academic  advisers  in  an 
intrusive  advisement  program  to  assist  students  in  rebuilding 
their  grade  point  average. 

What  are  you  trying  to 
preserve ? What  you 
have  and  want  to  keep. 

Identified  by  the  set  of  performance  standards,  methods  and 
repertoire  of  skills  that  is  currently  prevalent  among  the 
academic  advisors  that  help  students  to  improve  their 
academic  performance. 

What  are  you  trying  to 
avoid?  What  you  don’t 
have  and  don’t  want. 

Identified  by  the  skills  sets  and  methodology  used  by 
academic  advisers  in  the  academic  advisement  program  that 
are  ineffective  in  uplifting  the  academic  standards  of  students. 

What  are  you  trying  to 
eliminate ? What  you 
have  but  want  to  get  rid 

of. 

Identified  by  the  procedures,  skills  sets  and  methods  used  by 
academic  advisers  in  the  intrusive  academic  advisement 
program  that  is  not  beneficial  and  effective  towards  uplifting 
the  academic  standards  of  students. 

Nichols’  Goals  Grid  model  helped  us  to  develop  our  project  goal  which  helped  guide  our  decisions 
throughout  the  project.  Our  project  statement  clearly  stated  this  as  “This  performance  analysis  project  aims 
to  identify  exemplary  skills  among  academic  advisers  in  an  intrusive  advisement  program  for  at-risk 
students  and  identify  the  repertoire  of  skills  necessary  for  advisers  to  assist  students  in  re-huilding  their 
grade  average.  ” 

Human  Performance  Engineering  Model 

Human  Performance  Engineering  Model  (Gilbert,  1996)  formed  the  foundation  of  all  of  our  work 
in  this  project.  The  concepts  of  leisurely  theorems  and  the  Performance  Matrix  (Gilbert,  1996)  were  critical 
in  all  phases  of  our  performance  analysis.  We  tried  to  analyze  the  target  performance  of  advisers  during 
advisement  sessions  at  several  levels  of  vantage  in  the  Performance  Matrix.  These  factors  played  an  equally 
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important  role  in  the  actions  and  decisions  of  the  performer  during  the  advisement  sessions.  For  example, 
are  advisers  sensitive  to  cultural  differences  among  students,  are  they  empathetic  towards  students,  are  they 
being  resourceful,  etc.  Most  important  to  our  team,  the  levels  of  vantage  helped  us  narrow  our  scope  and 
allowed  us  to  be  more  effective  with  our  energy  and  time  as  we  conducted  the  performance  analysis. 
(Gilbert,  1996) 


Table  3:  Performance  Matrix  of  the  Academic  Advisors 


Vantage 

Level 

Accomplishment 

Variables 

Content 

Purpose 

Accomplishment 

Examples 

Philosophic 

Academic  advisement 

Students  and 

For  the 

Self  efficacy;  A 

al 

should  improve  the 
way  a college  student 
feels  about  being  in 
college 

advisers  will 
feel  good 
about  their 
experience 

good  of  the 
individual 

sense  of  well-being 

Cultural 

Academic  advisement 
will  improve  the  quality 
of  a student’s 
experience  in  college 

Students  will 
incorporate 
the  university 
into  their 
identity 

For  the 
good  of  the 
university 

A sense  of 
belonging 

Policy 

Academic  advisement 
will  retain  students  who 
were  at  risk  of 
expulsion 

Students  will 
improve  their 
grade  point 
averages 

For  the 
good  of 
both  the 
university 
and  the 
student 

A greater 
percentage  of 
students  returning 
to  Good  academic 
standing  from  the 
Intrusive 
Advisement 
Program 

Strategic 

Academic  advisement 
will  assist  students  with 
improving  their  grade 
point  averages 

Students  will 

develop 

strategies  for 

success; 

Advisers  will 

assist 

students 

For  the 
good  of 
both  the 
university 
and  the 
student 

Students  will 
demonstrate 
successful 
completion  of  their 
course  load  at  the 
end  of  each 
semester 

Tactical 

Academic  advisement 
will  assess  students 
individually  and 
develop  custom 
interventions  to  help 
students  with 
improving  their  grade 
point  averages 

Students  will 
develop  their 
personal 
strategies  for 
success; 
Advisers  will 
assist 
students 
individually 

For  the 
good  of 
both  the 
university 
and  the 
student 

Students  will 
demonstrate 
successful 
completion  of  their 
course  load  at  the 
end  of  each 
semester 

Logistic 

Academic  advisement 
will  take  place  in  a 
university  designed 
center  under  the 
guidelines  set  by  the 
university  using 
university  resources 

Advisers  will 
perform 
consistently 
and  use  the 
same  tools 
and 

resources 

For  the 
good  of 
both  the 
advisers 
and  the 
students 

Advisers  will 
complete  their 
responsibilities 
using  university 
resources 

Needs  Assessment  Model 

We  concentrated  upon  the  selection,  design  and  development  of  tools  and  methods  in  order  to 
determine  the  performance  gap  based  upon  what  we  had  learned  from  Allison  Rossett  (1998).  The  analysis 
tools  used  were  interviews  (stakeholders  and  subject  matter  experts),  surveys,  document  reviews  and 
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observations  of  performers.  This  was  performed  in  order  to  determine  the  optimals,  actuals,  feelings,  causes 
and  solutions  (Rossett,  1998)  as  revealed  in  Table  4. 


Table  4:  Needs  Assessment  in  the  Project 


Aspects  to 
Consider 

Questions  to  Answer 

Optimal  performance 

Optimal  performance  among  advisers  (who/what/how 
characterizes  the  exemplary  performance?) 

Actual  performance 

What's  going  on  among  the  academic  advisers?  What  are  the 
actual  performance  levels? 

Feelings 

How  do  performers  (advisers)  and  stakeholders  (department, 
students  and  university)  feel  about  the  academic  advisement 
program? 

Causes 

What  is  causing  the  performance  problems  among  academic 
advisers? 

Solutions 

Upon  performance  analysis,  what  suitable  solutions  should  be 
suggested  that  can  solve  the  performance  discrepancies  among 
academic  advisers? 

Critical  Factors  Affecting  the  Performance  of  Academic  Advisors 

We  focused  on  three  main  factors  that  influence  the  performance  of  the  adviser:  tools, 
performance  environment,  and  motivation  and  skills.  What  were  the  tools  available  to  the  academic 
advisor?  Were  the  tools  available  to  the  academic  advisor  relevant  to  his  performance?  Were  the  tools 
helping  the  advisers  to  gain  sufficient  feedback  from  the  stakeholders?  We  also  examined  the  supervision 
and  guidance  from  the  superiors  of  the  academic  advisor  in  some  critical  advisement  issues.  How  did  the 
factors  within  the  performance  environment  affect  the  performance  level  of  advisers?  Additionally,  we 
examined  the  motivation  level,  communication  skills,  empathetic  skills,  listening  skills,  resourcefulness 
(Gilbert,  1996)  of  the  academic  advisor.  Did  the  academic  advisor  possess  the  required  knowledge,  skills 
and  motivation  levels  required  to  perform  the  job? 

Based  on  a series  of  interviews  with  the  subject  matter  experts,  descriptive  surveys,  and 
stakeholders  and  on-site  observations  of  the  academic  advisor  advising  students,  a survey  using  1-5  scale 
(1,  the  least  important  to  academic  advisement  and  5,  the  most  important  to  academic  advisement)  was 
developed  and  conducted  among  all  the  academic  advisors  identified  a list  of  knowledge  and  skills  essential 
to  perform  academic  advising  to  help  student  improve  their  GPA. 

Table  5:  Essential  Knowledge  and  Skills  of  the  Academic  Advisor 


Knowledge 

Scor 

e 

Skills 

Scor 

e 

Understanding  the  responsibilities 
of  academic  advisor 

4.5 

Listening 

4.83 

Strategies  that  can  help  a student 
improve  their  GPA 

4.5 

Being  articulate  in  your  manner 
of  speaking 

4.5 

Time  Management 

4.33 

Paying  attention  to  details 

4.5 

University  rules,  regulations,  and 
policies 

4.33 

Disseminating  information 
accurately 

4.5 

Study  Skills 

4.17 

Performing  assessment 

4.5 

Degree  requirements,  options  with 
majors,  and  departmental 
requirements 

4.17 

Relating  complex  concepts  in  a 
concise  manner 

4.33 

How  to  customize  strategies  to 
meet  the  needs  of  a particular 
student 

4.17 

Having  patience 

4.33 

Techniques  for  balancing  work  and 
school  responsibilities 

4.5 

Identifying  circumstances 

4.17 

Strategies  that  can  help  a student 

4.5 

Identifying  patterns 

4.17 
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improve  their  GPA 

University  Resources 

3.5  Matching  appropriate 
interventions 

3.83 

Developing  custom  interventions 

3.83 

Taking  initiative 

3.67 

Being  persuasive 

3.33 

Listening  skills,  knowledge  about  how  to  balance  work  and  school  responsibilities,  and  a caring 
attitude  towards  the  student  are  the  critical  components  for  the  performer  to  perform  his/her  task.  As 
evidenced  by  their  stories,  each  adviser  takes  a similar  approach  to  assist  students  with  improving  their 
GPAs. 

Taking  the  time  to  hear  the  student  articulate  their  problems  with  their  GPAs  is  an  essential  part  of 
the  task.  Raising  the  student’ s awareness  of  issues  related  to  their  responsibilities  and  the  consequences  of 
their  actions  is  accomplish  through  a variety  of  techniques,  but  the  end  result,  to  be  successful,  requires  that 
the  student  buy  into  the  process  and  make  a commitment  to  improvement.  In  the  words  of  the  performers 
this  is  accomplished  through  analysis,  education,  reality  checks,  and  personal  attention. 

Analysis  Results  and  Recommendations 

The  results  of  performance  analysis  guided  by  ADDIE  instructional  design  model,  the  Goals  Grid 
Model  (Nichols,  2004),  Human  Performance  Engineering  Model  (Gilbert,  1996)  and  Analysis  Model  of 
actuals,  optimals,  feelings,  causes,  and  solutions  (Rossett,  1998)  identified  the  essential  knowledge  and 
skills  required,  and  standards  of  exemplary  performance  in  academic  advisement.  Accordingly, 
recommendations  were  made  to  enhance  the  performance  of  the  academic  advisors.  The  recommendations 
fell  into  two  categories  instructional  and  non-ins tructional. 

Instructional  Interventions 

Academic  Advisers  could  benefit  from  the  knowledge  about  how  their  individual  students  are 
progressing  on  a semester-by- semester  basis.  Developing  a job  aid  that  instructs  an  adviser  on  how  to  find 
this  information  in  the  database  of  student  records  is  an  inexpensive  intervention  that  would  enable  advisers 
to  monitor  their  student’s  progress.  To  meet  this  same  need  a training  workshop  could  be  implemented,  but 
a job  aid  would  be  a much  more  simple  and  effective  tool.  The  rationale  for  this  intervention  comes  from 
our  discovery  during  the  calculation  of  the  PIP  that  advisers  had  no  idea  how  many  of  their  students  return 
to  Good  standing  at  the  end  of  each  term. 

Some  advisers  may  not  understand  how  or  why  this  information  could  help  them  so  developing  a 
marketing  plan  for  a job  aid  like  this  would  be  essential.  If  this  intervention  is  selected  the  project  team 
recommends  that  key  change  agents  be  targeted  among  the  advisement  staff  as  champions  on  its  use.  When 
introducing  the  tool  a well  thought  out  description  of  how  and  why  it  should  be  used  would  also  aid  in  its 
adoption  and  assimilation. 

Non-Instructional  Interventions 

After  completion  of  the  document  review  for  the  project,  it  was  determined  that  advisers  cannot 
effectively  evaluate  their  job  performance.  For  example,  as  long  as  an  advisee/student  continues  at  the 
university  in  the  status  of  supervision  or  fails  to  probation,  the  adviser  may  continue  to  work  with  that 
student,  but  once  that  student  moves  to  good  academic  standing  or  is  excluded,  the  individual  student  data 
is  not  reported  to  the  adviser. 

A recommendation  for  evaluating  adviser  performance  would  be  to  develop  and  implement  an 
automatic  event  notification  database  linked  to  the  student’ s academic  record.  Once  a student  reaches 
supervision,  their  adviser  would  register  the  student  in  the  notification  database  and  would  input  their  own 
email  as  the  path  for  notification.  The  adviser  would  be  notified  through  email  when  either  of  the 
following  events  occurred:  the  student  moved  to  good  academic  standing  or  was  excluded.  Results  of  this 
notification  would  be  1)  Essential  feedback  for  advisers  in  evaluating  their  counseling  effectiveness  with 
individual  students,  and  2)  Data  from  the  notification  database  could  also  report  longitudinal  results  for  all 
advisers  in  the  department,  providing  and  indicating  how  specific  students  of  each  adviser  either  moved  to 
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good  standing  and  exclusion.  This  information  could  be  used  to  study  the  success  rate  of  advisers,  identify 
exemplars,  and  to  make  recommendations  for  instructional  strategies  for  improving  adviser  effectiveness 
and  performance. 


Conclusion 

Investigating  a performance  in  which  the  performers  are  delivering  an  affective  service  to  the 
target  audience  presents  some  unique  challenges.  Among  these  challenges  include  how  one  measures  the 
performance.  Where  quantitative  measures  lack,  qualitative  measures  proved  details  necessary  for  a human 
performance  technician  to  make  recommendations. 

Despite  the  project  team’s  best  efforts  to  quantify  the  performance  in  this  study,  in  the  end  the 
words  of  the  performers  themselves  provided  us  with  the  data  that  we  needed  to  clearly  understand  the 
performance.  From  another  perspective,  through  our  perceived  failure  to  pin  down  an  exemplary  performer, 
we  found  one  of  the  richest  pieces  of  data  during  our  assessment.  In  retrospect,  had  we  skipped  steps  in  the 
ADDIE  model  that  guided  our  work,  we  may  not  have  realized  the  full  extent  of  problem  - advisers  don’t 
know  how  many  of  their  students  are  returning  to  Good  standing  as  a result  of  their  work. 

This  presents  a significant  barrier  for  advisers  because  they  lack  feedback  that  could  add  potential 
value  to  the  way  they  approach  each  new  at-risk  student.  Despite  this  barrier,  Academic  Advisers  in  the 
University  Advisement  Center  appear  clear  in  their  understanding  of  the  expectations  of  their  Director  and 
Associate  Director  as  evidenced  by  their  consensus  on  100%  of  the  concepts  that  emerged  from  the 
interviews  with  the  stakeholders.  The  data  from  the  survey  both  detail  the  performance,  but  also  is 
testimony  to  a mutual  understanding  between  the  performers  and  the  stakeholders  about  the  performance. 

The  recommendations  from  the  project  team  reflect  the  data  that  we  discovered  using  a systematic 
process  to  investigate  the  performance  of  Academic  Advisers  helping  students  to  improve  their  GPAs. 
Additional  value  at  the  policy  level  to  potentially  be  gained  by  choosing  one  of  the  interventions 
recommended  could  be  improved  retention  at  the  university  in  which  this  performance  takes  place. 
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Lived  Experience  of  the  TEEMS  Students’  Collaboration  in  Electronic 
Portfolio  Construction:  A Study  Framework 

Charles  Xiaoxue  Wang 
Georgia  State  University 

Abstract 

This  is  an  ongoing  phenomenological  study.  The  study  examines  lived  experience  of  students  in  the 
Teacher  Education  Environments  in  Mathematics  and  Science  (TEEMS)  Programs  when  collaborating 
with  their  peers  in  an  electronic  portfolio  construction  in  a technology  integration  course  at  a southeast 
state  university.  This  paper  shares  with  the  readers  a phenomenological  study  framework  used  in  mining 
the  meaning  of  the  TEEMS  student  collaboration.  It  describes  the  study  background,  offers  research 
questions  and  rationale,  outlines  the  study  processes  and  methods  and  share  with  the  readers  its  findings. 

Introduction 

The  Teacher  Education  Environments  in  Mathematics  and  Science  (TEEMS)  Programs  are 
alternative  paths  to  initial  teacher  certification  at  a southeast  state  university.  With  a middle  and  secondary 
focus,  the  TEEMS  Programs  require  45  credit  hour  at  the  graduate  level  and  full  time  enrollment.  Students 
entering  TEEMS  Program  must  have  an  undergraduate  degree  in  their  content  field  and  must  pass  a 
rigorous  selection  and  interview  process  for  admission.  Admission  occurs  once  a year  in  each  content  field 
to  form  cohort  groups.  Students  who  finish  the  TEEMS  Programs  study  will  obtain  a master  of  education 
degree  and  recommendation  for  initial  teacher  certification  in  the  state  (Shoffner,  2005).  Though  expanded 
to  include  the  Certification  Program  for  Reading,  Language,  and  Literacy  (RLL)  and  the  Certification 
Program  for  Teaching  English  as  a Second  Languages  (ESOL),  these  alternative  preparation  programs  are 
still  referred  to  by  their  originally  acronym  name:  TEEMS  Programs. 

No  Child  Left  Behind  Act  of  2001  presents  a great  challenge  to  the  whole  society  to  help  every 
child  to  achieve  at  a higher  level  and  prepare  them  for  the  future  (U.S.  Department  of  Education,  2002).  It 
recommends  that  by  the  eight  grade  all  students  should  be  technologically  literate  and  it  references 
technology  as  a very  important  source  of  support  for  teaching  and  learning  across  the  curriculum.  To  better 
prepare  teacher  force  to  meet  this  challenge,  many  teacher  education  programs  include  educational 
technology  courses  in  their  program  study.  Many  of  these  educational  technology  courses  are  designed 
according  to  NCATE  and  ISTE  standards  and  are  designed  to  increase  effective  use  of  technology  in  K-12 
schools. 

The  Middle-Secondary  Education  and  Instructional  Technology  (MSIT)  Department  at  a southeast 
state  university  offers  multiple  sessions  of  a technology  integration  course  (IT  7360:  Technology  for 
Educators)  for  its  students  in  the  TEEMS  Programs  every  semester.  The  course  was  designed  to  help  the 
TEEMS  students  to  meet  the  National  Educational  Technology  Standards  for  Teachers  (NETS)  by  the 
International  Society  of  Technology  in  Education  (ISTE)  (Shoffner,  2005).  IT  7360:  Technology  for 
Educators  is  a required  three  credit  hour  course  that  prepares  the  TEEMS  students  with  essential  skills  and 
knowledge  to  integrate  technology  in  their  instructions.  The  course  requires  the  TEEMS  students  to 
complete  a series  of  assignments  that  involve  using  technology  to  solve  instructional  problems.  Also  the 
TEEMS  students  are  required  to  complete  three  structured  reflection  papers  at  the  beginning,  mid  point  and 
at  the  end  of  the  semester.  In  addition  to  this,  the  TEEMS  students  must  construct  an  electronic  portfolio  to 
demonstrate  their  competency  in  instructional  design  and  technology  integration.  The  electronic  portfolio 
includes  student  professional  development  materials,  and  a technology  enhanced  learning  environment  in 
his/her  subject.  The  learning  environment  consists  of  an  instructional  unit  plan,  sample  lesson  plans, 
student-centered  learning  activities,  and  assessment  plans  (Shoffner,  2005). 

Problem  Statement 

Three  surveys,  Technology  Proficiency  Self  Assessment  Survey,  Teacher’s  Attitudes  toward 
Computers  and  Student  Information  Survey  were  administered  in  IT  7360  at  the  beginning  of  each 
semester.  Technology  Proficiency  Self  Assessment  Survey  was  created  by  and  used  with  permission  of  Dr. 
Margaret  Merlyn  Ropp  (2005).  The  Student  Information  Survey  collected  information  about  the  TEEMS 
students’  basic  professional  experiences,  computer  proficiencies,  their  knowledge  of  technology  standard, 
and  their  stage  of  adoption  of  technology  in  their  instructions.  The  results  of  these  surveys  and  observations 
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by  the  course  instructors  indicated  that  the  technology  proficiency  of  the  TEEMS  students  was  low  and 
their  knowledge  of  technology  integration  was  limited  at  the  beginning  of  the  course. 

The  low  technology  proficiency  and  the  limited  understanding  of  the  theoretical  framework  behind 
technology  integration  presented  a daunting  challenge  to  the  course  instructors  of  IT  7360.  One  effective 
way  of  overcoming  such  a challenge  was  to  create  an  active  learning  community  among  the  TEEMS 
students  by  capitalizing  on  their  shared  knowledge  and  expertise.  Over  the  past  three  semesters,  course 
instructors  encouraged  the  TEEMS  students  to  collaborate  with  their  peers  when  constructing  their 
electronic  portfolios.  The  documented  observations  of  instructors  as  well  as  the  responses  of  students  in 
their  reflection  papers  suggested  that  the  collaborations  among  the  TEEMS  students  when  constructing 
their  electronic  portfolios  was  an  important  phenomenon  and  need  a systemic  study  to  reveal  its  essence. 
This  phenomenological  study  was  to  describe  collaborative  learning  experiences  when  constructing  their  E- 
Portfolio  using  the  participants’  own  words  to  reveal  their  true  and  lived  experience. 

Research  Questions  and  Rationale 

Three  major  questions  guided  this  study.  (l)What  is  lived  experience  of  the  TEEMS  students 
when  collaborating  with  their  peers  in  the  construction  of  their  electronic  in  a technology  integration 
course?  (2)  How  might  knowledge  generated  from  the  answers  to  the  question  be  used  to  help  the  TEEMS 
students  collaborate  more  effectively  in  their  electronic  portfolio  construction?  (3)How  might  the  answers 
to  the  question  be  used  to  help  IT  7360  instructors  facilitate  effective  collaboration  during  electronic 
portfolio  construction? 

Underneath  the  first  major  questions,  a set  of  questions  were  asked. 

• What  are  criteria  the  TEEMS  students  used  in  selecting  their  partner  to  collaborate  in  the 
electronic  portfolio  construction? 

• What  are  attributes  and  characteristics  do  the  TEEMS  student  use  to  describe  their 
partners  in  collaboration? 

• What  are  things  the  TEEMS  students  do  that  make  them  feel  they  are  collaborating  in 
electronic  portfolio  construction? 

• What  roles  do  the  TEEMS  students  assume  when  working  collaboratively  in  electronic 
portfolio  construction? 

• What  expectations  do  the  TEEMS  student  have  for  each  other  in  collaboration 
(responsibilities,  portfolio  contents,  task  deadlines,  and  the  effectiveness  of 
collaboration)? 

• What  factors  do  the  TEEMS  students  describe  that  facilitated  their  collaboration? 

• What  factors  do  the  TEEMS  students  describe  that  impeded  to  their  collaboration? 

• What  are  the  most  exciting  moments  during  their  collaborations  in  electronic  portfolio 
construction? 

• What  are  the  most  frustrated  moments  during  their  collaborations  in  electronic  portfolio 
construction? 

• What  are  the  interesting  events  happened  during  their  collaboration  in  electronic  portfolio 
construction? 

• What  factors  do  the  TEEMS  students  contribute  to  their  sense  of  satisfaction  upon 
completion  of  the  portfolio? 

• What  factors  do  the  TEEMS  students  contribute  to  dissatisfaction  upon  completion  of  the 
portfolio? 

Collaboration  refers  to  the  phenomenon,  the  intuitive  experience,  shared  when  the  TEEMS 
students  worked  together  to  achieved  the  pre-determined  aim  of  constructing  an  electronic  portfolio.  It  is 
the  critical  phenomenon  under  the  study  and  its  units  of  analysis  are  collaboration  experiences  of  the 
individual  TEEMS  students  in  IT  7360  course.  The  site  of  study  is  at  the  College  of  Education  in  a 
southeastern  state  university. 

Collaboration  as  a social  phenomenon  is  complex  and  frequently  misunderstood  (Youtz,  1997).  As 
a result  of  varied  and  multiple  interpretations  regarding  what  defines  collaboration,  it  is  little  wonder  that  so 
many  studies  on  collaborative  learning  exist.  Specifically  for  this  study,  the  researcher  wants  to  examine  his 
own  TEEMS  students’  collaboration  when  constructing  their  electronic  portfolios  to  better  understand  the 
essence  of  this  phenomenon  as  described  by  their  own  words.  This  study  by  its  nature  was  limited  to  the 
TEEMS  students  who  collaborate  with  their  peers  in  electronic  portfolio  construction  in  IT  7360  course. 
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The  purpose  of  this  investigation  is  to  determine  what  constitutes  the  essential  experience  of  the 
TEEMS  students  who  collaborate  in  their  electronic  portfolio  construction.  According  to  Van  Manen 
(1990),  phenomenological  research  is  the  study  of  essence  of  phenomena  by  mining  and  describing 
experiential  meaning  as  people  live  them.  By  revealing  the  essence  of  collaboration  among  the  TEEMS 
students,  the  investigation  should  provide  stronger  evidence  for  the  employment  of  collaborative  learning 
in  the  IT  7360  course  as  a means  of  facilitating  learning  and  the  construction  of  the  electronic  portfolios. 

The  Process  and  Methods 

The  phenomenological  approach  (Creswell,  1998)  was  selected  for  this  study  because  the  intent 
was  to  better  understand  the  meaning  of  “lived  experiences”  of  the  several  individual  TEEMS  students  in 
IT  7360  course.  If  phenomenology  mines  the  meaning  of  lived  experience,  this  phenomenological  approach 
employed  here  examines  the  TEEMS  students’  lived,  conscious  experiences  and  uncovers  what  constitutes 
the  essential  features  and  essence  of  their  lived  experiences  (Van  Manen,  1990). 

Figure  1 : The  Study  Timeline,  Data  Sources  and  Study  Process  reveals  the  design  of  this  study. 

The  study  was  started  in  fall  2004.  The  major  means  used  for  this  study  are  two  interviews  in  addition  to 
the  TEEMS  students’  reflection  papers  and  two  sets  of  surveys  done  at  the  beginning  and  at  the  end  of  the 
semester.  The  first  interviews  were  well- structured  and  the  same  questions  were  asked  to  all  the 
participants.  The  interviews  were  conducted  two  weeks  after  the  TEEMS  students  started  their 
collaboration  in  the  electronic  portfolio  construction  right  after  their  mid-term  reflection  paper.  The 
interviews  were  a little  over  20  minutes  on  average.  The  purposes  of  the  first  interviews  were  to  collect  data 
about  the  TEEMS  student  collaboration  and  their  feel  about  the  collaboration  partners.  The  first  interviews 
also  provided  possible  and  crucial  information  to  customize  interview  questions  for  the  second  interviews 
in  order  to  facilitate  deeper  and  richer  reflection  and  description  of  their  collaboration  experiences  during 
the  interviews  in  the  second  stage. 

Figure  1:  The  Study  Timeline,  Data  Sources  and  Study  Process 


The  second  interviews  were  conducted  two  weeks  after  they  finished  their  course.  The  second 
interviews  focused  more  on  describing  their  experiences  of  collaboration  as  a reflective  process. 
“Phenomenological  reflection  is  not  introspective  but  retrospective.” (V an  Manen,  1990,  p.10  original 
emphasis)  For  this  reason  semi- structure  interviews  were  used  which  asked  each  participants  a set  of  fixed 
questions  as  well  as  some  customized  and  improvised  questions  during  the  interview  to  facilitate  reflection 
and  to  uncover  the  essence  of  their  collaboration.  The  second  interviews  were  about  one  hour  on  average. 

Four  TEEMS  students  participated  in  this  study  in  the  fall  2004.  They  all  signed  the  informed 
consent  forms  after  the  researcher  carefully  explained  the  nature  and  the  process  of  this  study.  The 
explanations  of  the  study  focused  on  the  confidentiality.  They  were  ensured  and  understood  that  all 
interviews  were  conducted  individually  and  their  interview  content  would  remain  confidential  and  would 
not  be  revealed  to  others  including  their  collaboration  partners.  Each  interview  was  taped,  transcribed  into 
text,  and  then  analyzed. 

Four  TEEMS  students  participated  in  this  study.  They  all  entered  into  the  same  TEEMS  program 
majoring  in  Math  Education  in  the  summer  of  2004.  Among  the  four  TEEMS  students,  one  was  a pre- 
service teacher  and  other  three  were  in-service  teachers.  The  pre- service  teacher  did  not  have  any  formal 
classroom  teaching  experience  while  the  three  in-service  teachers  had  been  teaching  math  on  provisional 
status  since  2003.  Two  in-service  teachers,  who  collaborated  on  their  electronic  portfolio  construction, 
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worked  in  the  same  department  of  the  same  school  public  high  school  and  had  known  each  other  since 
Spring  2003.  These  two  TEEMS  students  collaborated  whenever  possible  both  in  their  work  at  school  and 
in  the  study  at  the  TEEMS  programs. 


Preliminary  Findings 

This  study  is  still  on-going.  Presented  below  are  the  preliminary  findings.  The  themes  emerged 
from  data  analysis  of  the  second  interviews  revealed  a few  interesting  insights  of  the  TEEMS  students’ 
collaboration  with  their  peers  in  their  electronic  portfolio  construction. 

Defining  Collaboration 

Although  the  study  participants  had  their  own  definitions  about  their  collaboration  in  electronic 
portfolio  construction,  they  all  agreed  that  it  was  serious  undertaking  with  “a  big  task”  and  pressures  of 
“deadlines.”  The  participants  did  not  talk  much  about  responsibilities,  roles,  and  goals  as  the  researcher  had 
expected  reflecting  upon  their  collaboration.  However,  they  defined  their  collaboration  in  electronic 
portfolio  construction  as  basically  “working  together”  to  get  things  done,  “to  solve  your  problems”,  “fun” 
and  “part  of  our  life”,  “things  we  do  often. . .”,  “wonderful  learning  experience”.  The  study  participants 
offered  very  similar  definitions  of  their  collaborations. 

“Well,  it’s  simply  working  together,  you  know,  to  get  things  done.  Creating  our  portfolio, 
you  know,  was  a big  task  and  . . . Working  with  [name  omitted]  made  things  much  easier. 

You  don’t  have  to  always  stand  on  your  own.  Just  like  you  have  your  buddy....  working 
together,  especially  the  one  that  you  have  collaborated  before  and  you  know  [name 
omitted]  is  good  on  this.”  (Interviewee  A,  p.  3) 

“Working  together  to  solve  your  problems. . . It’s  fun  and  it’s  part  of  our  life,  things  we 
do  often.  We  always  collaborate  in  our  math  courses  and  it  was  just  another  wonderful 
learning  experience  with  her. . .”  (Interviewee  B,  p.  4) 

“Collaboration  is  a way  to  get  busy  and  crazy. . . As  a matter  of  fact,  we  accomplished  a 
lot  each  time  we  work  together. . .but  sometimes,  especially  when  you  see  the  deadline 
was  just  around  the  corner,  Oh,  man. . .’’(Interviewee  D,  p.3) 

Collaboration  Partner  and  Positive  Pre- Collaboration  Experience 

All  the  study  participants  enjoyed  their  collaborations  and  spoke  highly  of  their  collaboration 
partner.  The  study  participants  had  very  positive  experiences  of  collaborating  with  their  partners  before 
their  collaboration  in  their  electronic  portfolio  construction  was  just  “things  we  do  often. . .”  Their  positive 
comments  on  their  collaboration  partners  revealed  that  their  satisfactions  of  collaboration  and  also  indicated 
the  importance  and  impact  of  the  positive  pre-collaboration  experience  with  their  partners  on  their 
collaborations  in  electronic  portfolio  construction. 

“We  collaborate  in  our  work  at  school  and  in  our  study  at  Georgia  State  [University].  She 
is  my  buddy  and  she  is  so  quick  on  things  and  always  has  a lot  of  wonderful  ideas.  I knew 
we  would  do  a great  job!,  I mean  in  portfolio”  (Interviewee  A,  p.7) 

“He  is  very  professional,  always  on  the  tops  of  things,  very  detail  oriented,  trustworthy 
and  easy  to  work  with.  We  worked  together  before. . . we  are  very  different  but  we  like 
each  other.  I enjoyed  working  with  him. . .he’s  a great  guy  to  work  with.”  (Interviewee  C, 

P-7) 

“We  did  two  big  course  projects  last  summer  and  it  worked  quite  well. . .She  always 
knows  what  to  do  and  always  get  things  done  fast.  . . .In  an  intensive  program  like  Math 
TEEMS,  you  want  to  work  with  a colleague  like  her. ’’(Interviewee  D,  p.9) 

Unique  Ways  of  Collaboration 

Two  study  participants  worked  in  the  same  school  and  lived  close  to  each  other  but  quite  a 
distance  from  the  University.  A lot  of  collaboration  work  was  done  on  the  way  to  the  University  and  on  the 
way  back  home.  Their  descriptions  depicted  an  interesting  picture  that  mirrored  off  their  collaboration 
experiences 

“We  designed  our  layout  that  evening  at  Georgia  State  [University].  The  we  revised  the 
layout  on  the  drive  home,  We  rode  together  to  school. . . Entire  drive  home,  we  completed 
the  page  layout  and  divided  the  [web]  pages  to  work  on. . . the  drive  was  about  40  to  60 
minutes,  we  discussed  portfolio  all  the  way  home.”  (Interviewee  A,  p.  15) 
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“We  also  discussed  our  assignment  on  the  way  home,  we  spoke  so  loud  in  the  car. . . We 
also  planned  our  assignments  for  the  class  around  the  portfolio  so  that  we  had  a really 
cool  Polynomials  theme  that  we  can  now  use  in  our  classrooms.”  (Interviewee  A,  p.16) 

“As  much  as  I hated  driving  to  Atlanta,  the  drives  seemed  to  allow  us  to  get  so  much 
done.  Sometimes  we  would  talk  about  other  things,  but  in  the  car  proved  to  be  the  most 
convenient  time  for  us  to  think  aloud  and  discussed  our  ideas.  I also  enjoyed  every 
minutes  of  working  with  [name  omitted].  We  were  able  to  help  each  other  and  improve 
our  teaching  abilities  and  give  each  other  wonderful  ideas.  It  always  made  the  driving 
home  so  fast”  (Interviewee  B,  p.15) 

Exciting  moments 

We  felt  the  favor  of  excitement  of  collaboration  in  above  description.  Yet  the  more  obvious 
exciting  moments  were  those  when  they  were  “in  the  zone,”  when  they  saw  “the  light  at  the  end  of  the 
tunnel,”  and  when  they  saw  their  great  accomplishment. 

“We  felt  that  we  were  in  the  zone. . .Everything  just  fits  in  . . . we  knew  we  would  finish  it 
that  day  and  the  portfolio  was  great,  all  of  our  documents  were  in  place  and  links  were 
fine,  everything  was  fine,  we  were  in  the  zone,  in  the  zone,  you  know!”  (Interviewee  D, 

p.  21) 

“Exciting  moments. . . The  day  when  we  finished  our  portfolio.  It  was  a great  day,  and  I 
was  in  the  lab  first.  [Name  omitted]  came  in  late  and  I remember  it  was  Saturday. . .He 
asked  me  ‘Are  you  ready?  Let’s  finish  it  today.’  He  sounded  so  confident  and  . . . Almost 
instantly,  um. . . I saw  a light  at  the  end  of  the  tunnel.  That  moment. . .was  so  exciting! ! 

That  day  was  just  a great  day!”  (Interviewee  C,  p.20) 

“I  showed  our  portfolio  to  my  students,  my  daughter  and  husband  and  I also  showed  it  to 
my  colleagues  at  school.  I got  great  response  like  ‘Good  grief,  how  much  did  you  guys  do 
this  semester.’  It  was  exciting  when  you  heard  comments  like  this  and  you  know  that  you 
did  a great  job.  It’s  a big  accomplishment.  I was  extremely  proud. . .”  (Interviewee  B, 

P-22) 

“We  checked  all  the  links  and  images  on  our  portfolio  on  another  computer.  At  that 
moment,  the  portfolio  suddenly  became  mine.  We  worked  on  it  for  weeks  and  I felt  so 
weird  that  it  suddenly. . . like  your  baby  was  born.  It  was  the  most  exciting  and  it  was 
weird  too. . .Anyway,  it  was  exciting.  The  portfolio  site  was  extremely  comprehensive 
and  others  could  definitely  use  it.”  (Interviewee  A,  p.23) 

Frustrations 

In  describe  their  frustrations  experienced  during  their  collaboration  revealed  the  obstacles  that 
impeded  their  collaboration  and  should  get  our  attention.  These  frustration  moments  indicated  that  the 
collaboration  format,  tools,  individual  working  styles,  and  even  the  physical  well  being  were  important 
factors  contributed  to  their  frustration. 

“Definitely,  I hope  to  have  more  visualized  collaborations. . . synchronized  but  visualized 
when  we  were  working  at  home.  We  talk  a lot  on  our  cell  phones.  But  really  hope  to  see 
. . .1  am  a visual  learner.  I need  sometimes  to  see  what  was  on  her  screen. . . If  I could  do 
that,  some  problems  we  talked  for  hours  could  be  solved,  you  know,  just  one  click.” 
(Interviewee  D,  p.  16) 

“The  E-Locker  (a  web  format  file  transferring  protocol  that  university  offers  to  all  the 
students  to  access  their  files  on  university  network  drive)  was  not  user-friendly,  we  tried 
a few  times  to  upload  our  portfolio  files  through  it.  Oh,  it  was  horrible.  We  ended  up  with 
broken  links,  and  missing  images,  and  our  pictures  were  gone. . . And  we  decided  that  we 
need  to  work  together  at  the  computer  lab  downstairs.”  (Interviewee  C,  p.l  1) 

“Find  a good  chuck  of  time  to  work  together  was  always  a problem.  Let’s  me  take  that 
back,  it’s  not  really  a problem  but  it’s  difficult.  Difficult,  you  know  what  I mean.  I need  a 
big  chuck  of  time  to  work  on  it.  It’s  just  me.  I can  not  work  30  minutes  on  the  portfolio 
and  shift  gears  to  work  on  something  else  for  another  hour  and  then  come  back  to  work 
on  portfolio  again.  It’s  just  not  me.”  (Interviewee  D,  p.  14) 
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“Sometimes  we  were  so  tired  after  our  evening  class  when  we  met  in  the  lab.  We  sat 
there  doing  nothing  ...  I was  tired,  I wanted  to  go  home,  and  sleep.”  (Interviewee  C, 
P-16) 


Conclusion 

Though  the  results  presented  above  are  preliminary,  it  already  begins  to  present  us  a real  picture  of 
the  TEEMS  students’  collaboration  when  constructing  their  electronic  portfolio  in  the  technology 
integration  course  and  its  collaboration  essence.  “By  essence  we  do  not  mean  some  kind  of  mysterious 
entity  or  discovery,  nor  some  ultimate  core  or  residue  of  meaning. . .A  good  description  that  constitutes  the 
essence  of  something  is  construed  so  that  the  structure  of  a lived  experience  is  revealed  to  us  in  such  a 
fashion  that  we  are  now  able  to  grasp  the  nature  and  significance  of  this  experience  in  a hitherto  unseen 
way.”  (Van  Manen,  1990,  p.39)  This  study,  through  the  reflection  on  the  positive  collaboration  experience, 
also  confirms  the  usefulness  of  social  cognitive  learning  theory,  a theory  that  advocates  learning  through 
collaboration  in  the  real  social  contexts  (Driscoll,  2000).  Additionally,  it  adheres  to  theories  regarding 
distributed  cognition,  a premise  that  suggests  “learning  is  not  an  individual  act,  even  when  it  undertaken 
alone”  (Duffy  & Cunningham,  1996,  p.180)  and  “knowledge  does  not  reside  with  an  individual  learner 
alone”  (Gunawardena  & Mclsaac,  2004,  p.376)  but  is  distributed  among  members  of  the  community.  The 
new  picture  painted  by  the  participants’  own  words  the  researcher  got  in  this  study  will  shed  a light  on  the 
efforts  to  facilitate  learning  through  collaboration  as  a course  technology  integration  course. 
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If  you  build  it,  will  they  come?  Designing  an  instructional  computer 
game  for  an  undergraduate  computer  course. 


William  R.  Watson 

Indiana  University-Purdue  University  Indianapolis 

Introduction 

This  presentation  will  detail  the  process  and  experience  of  designing  Lifecycle , an  instructional, 
web-based  computer  game  for  my  undergraduate  Systems  Analysis  and  Design  course.  I will  briefly 
present  the  game,  discuss  the  process  of  designing  the  game,  present  the  underlying  rules  behind  the  game, 
present  initial  feedback  from  my  students,  and  share  lessons  learned.  This  presentation  should  be  of  interest 
to  anyone  considering  the  design,  development,  or  application  of  video  or  computer  games  to  education. 

Why  a computer  game? 

Video  and  computer  games  have  rapidly  become  a leading  form  of  entertainment  in  America  and 
across  the  world.  Huge  video  game  sporting  conventions  have  sprung  up  in  California  and  Seoul,  among 
other  cities,  and  players  regularly  compete  with  each  other  in  local  “LAN  parties.”  The  NPD  group 
reported  that  computer  and  video  game  software  sales  exceeded  a record  $10.3  billion  in  the  US  in  2002, 
slowing  but  remaining  strong  with  $10  billion  in  2003  (The  NPD  Group  Reports  Annual  2003  U.S.  Video 
Game  Industry  Driven  by  Console  Software  Sales,  2004).  This  compared  to  $9.5  billion  in  box  office  sales 
for  the  US  movie  industry  in  2002  (U.S.  Entertainment  Industry:  2002  MPA  Market  Statistics). 

With  the  emergence  of  video  and  computer  games  as  popular  entertainment,  it  is  only  natural  that 
educators  would  turn  to  this  new  media  to  examine  its  potential  for  use  as  instruction.  More  and  more 
researchers  have  begun  to  examine  video  and  computer  games  and  many  have  found  that  the  media  holds 
great  promise  for  engaging  instruction  (Reigeluth,  1999a;  Prensky,  2001;  Molenda,  2002;  Gee,  2003; 
Squire,  2003;  Aldrich,  2004).  More  and  more  instructors  are  seeing  that  their  students  are  demanding  more 
engaging  instruction,  and  video  games  seem  like  a strong  candidate  since  the  students  are  clearly  playing 
them.  Jenkins  reports  that  while  88  percent  of  surveyed  incoming  MIT  students  had  played  games  before 
the  age  of  10,  more  than  75  percent  were  still  playing  games  at  least  once  a month. 

However,  is  it  truly  realistic  for  an  instructor  to  design  an  educational  computer  or  video  game  for 
his  class?  I decided  to  test  this  for  myself  and  began  designing  a game  with  the  hopes  that  it  not  only  would 
be  more  engaging  for  the  students  but  that  it  might  actually  improve  their  understanding  of  the  relationship 
between  SAD  tools  and  concepts  that  I teach  in  my  class.  While  I am  continuing  to  refine  the  game  before 
testing  the  impact  on  the  students’  understanding,  I found  that  it  is  definitely  possible  to  design  an 
instructional  game  and  have  it  developed.  A senior  in  computer  graphics  at  my  university  took  on  the 
development  of  the  game  as  a senior  project,  and  it  was  entirely  designed  and  completed  in  under  a year. 

The  game 

Lifecycle  is  an  interactive  simulation  game  which  seeks  to  develop  an  understanding  of  Systems 
Analysis  and  Design  (SAD)  concepts  using  the  UML  methodology.  Players  will  play  the  role  of  a systems 
analyst  in  charge  of  a development  team.  The  development  team  is  seeking  to  develop  a quality  product  for 
the  Client  Corporation  using  very  various  UML  tools  to  document  the  system’s  requirements  and  design 
specifications.  Players  will  be  scored  based  on  the  quality  of  the  system,  time  spent  developing  the  product, 
and  user  satisfaction.  Other  variables  will  be  stored  which  will  impact  the  scoring  of  these  key  variables. 
The  goal  of  the  game  is  to  develop  the  highest  quality  system,  in  the  least  amount  of  time,  with  the  highest 
customer  satisfaction.  The  game  will  produce  a log  of  the  player’s  name,  player  actions  throughout  the 
game,  and  the  player’ s final  score. 

I designed  the  game  so  that  my  students  would  be  able  to  discover  underlying  UML  and  SAD 
concepts  as  well  as  the  relationship  between  these  concepts  and  the  hands-on  tools  they  use  to  document 
system  requirements  through  experimentation,  feedback,  and  yes,  “play.”  The  focus  of  the  game  is  on 
instruction  and  simulating  the  long  process  of  analyzing,  designing,  and  developing  a system.  I envisioned 
the  game  scaffolding  knowledge  presented  previously  in  class.  Therefore,  the  students  would  receive 
instruction  through  lecture  and  utilize  specific  UML  diagrams  in  class  projects.  Lifecycle  is  then  to  be 
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played  following  this  initial  introduction  to  the  material  so  that  the  students  can  experiment  as  analysts  and 
receive  instant  feedback  on  their  choices.  UML  focuses  on  a lifecycle  of  iterations,  meaning  diagrams  are 
created  and  revised  multiple  times.  The  overall  lifecycle  of  the  class  project  and  indeed  most  projects  is 
often  too  long  for  the  students  to  be  able  to  easily  make  the  conceptual  connections  that  I hope  they  will 
make.  Lifecyle  hypothetically  will  give  them  the  opportunity  to  repeatedly  go  through  a complete  system 
development  process  in  a short  amount  of  time  and  have  a unique  experience  each  time,  while  receiving 
important  feedback  that  will  allow  them  to  make  stronger  conceptual  connections  while  also  having  factual 
knowledge,  such  as  UML  and  SAD  terminology,  reinforced.  While  the  game  design  was  a daunting  task,  I 
believe  that  this  experience  has  shown  that  it  is  possible  for  an  individual  instructor  to  design  and  have 
developed,  an  instructional  game  that  students  will  enjoy  and  find  meaningful. 

The  design  process 

I had  several  goals  in  mind  when  approaching  the  task  of  designing  Lifecycle.  I wanted  the  game 
to  be  playable  via  the  Web  so  that  my  distance  students  would  also  be  able  to  use  the  game.  I also  wanted 
the  game  design  to  be  playable  in  paper  format  so  that  I could  test  the  underlying  structure  of  the  game  for 
accuracy  prior  to  having  the  game  actually  built.  I wanted  the  game  content  to  reinforce  specific  conceptual 
and  factual  instructional  objectives. 

I also  had  several  restrictions  that  would  place  limits  on  the  game’s  design.  As  I would  have  to 
rely  on  a single  student  to  develop  the  game,  it  would  need  to  be  simple  enough  that  this  was  possible.  I 
also  would  only  have  the  student’s  help  for  one  semester,  so  the  design’s  complexity  would  also  need  to  be 
simple  enough  to  be  completely  built  in  a single  semester.  Because  of  this,  Lifecycle  is  primarily  a text- 
based  game.  However,  it  does  utilize  images,  and  I am  currently  working  with  a second  student  to 
incorporate  more  audio  and  refine  the  visual  design  of  the  game. 

After  recognizing  both  my  primary  objectives  and  the  limitations  that  I was  working  under,  I set 
about  trying  to  figure  out  how  I would  create  an  underlying  structure  of  rules  that  would  cause  the  game  to 
behave  as  needed  and  reinforce  the  instructional  objectives  that  I had  decided  on.  I had  never  designed  a 
game  before,  so  I began  by  reading  a number  of  books  on  game  design.  By  far  the  most  helpful  resources  I 
had  were  Salen  and  Zimmerman’s  Rules  of  Play  (2003)  and  an  interview  with  Dr.  Michael  Molenda  of 
Indiana  University  (personal  communication,  February  25,  2005).  Rules  of  Play  is  very  helpful  to  the 
novice  game  designer  because  it  breaks  down  the  structural  design  (rules)  of  a game  (usually  a board  game) 
at  the  end  of  each  chapter.  This  helps  you  see  how  game  designers  tie  contextual  and  conceptual  objectives 
to  game  rules  that  will  reflect  these  objectives.  Dr.  Molenda  had  designed  an  instructional  board  game 
several  decades  ago  that  tried  to  simulate  the  process  of  managing  the  introduction  of  change  in  a school 
system.  He  went  over  the  game  and  how  the  dice  the  players’  rolled  determined  game  response  based  on  a 
chart  of  percentages  which  was  tied  to  game  feedback  and  action  outcomes.  This  game  has  since  been 
translated  into  a Web-based  game  by  Dr.  Ted  Frick  and  a group  of  his  graduate  students  (2005);  although, 
access  is  currently  limited  to  those  on  the  IU  system. 

After  reading  Rules  of  Play,  talking  with  Dr.  Molenda,  and  playing  the  Web-based  Diffusion 
game,  I had  a stronger  understanding  of  how  to  structure  my  game  so  that  it  could  incorporate  random 
occurrences,  create  variable  player  experiences,  and  still  promote  the  learning  objectives  that  I was 
interested  in.  I first  developed  the  context  of  the  game,  the  design  situation  and  the  system  stakeholders.  I 
then  determined  how  I would  score  the  game  and  translated  this  into  scoring  variables  which  would  track 
player  performance.  Next,  I created  a list  of  player  actions,  some  immediately  available,  others  only 
available  after  certain  conditions  had  been  met  and  decided  how  these  actions  would  impact  the  scoring 
variables  as  well  as  reflect  the  learning  objectives  I had  in  mind.  Finally,  I created  a list  of  random  impacts 
and  tied  many  of  these  to  player  actions  and  learning  objectives,  so  that  even  though  the  impacts  were 
random,  they  reinforced  learning  objectives  and  could  be  somewhat  controlled  by  player  action.  I detailed 
feedback  for  all  player  actions  and  random  impacts,  play-tested  the  design  and  wrote  up  a design  document 
which  was  given  to  the  student  developer.  I met  each  with  the  developer  to  review  the  process  of  the  game 
and  help  her  to  understand  the  underlying  rules  that  shaped  and  supported  the  game  and  its  use  as 
scaffolding  for  learning. 


The  game  structure 

Variables 

As  examining  the  underlying  structure  of  other  games  was  most  helpful  to  me  in  designing 
Lifecycle,  I will  try  to  briefly  present  Lifecycle’s  structure  and  how  it  reflects  my  instructional  goals  for  the 
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game.  I will  do  this  by  describing  the  game’s  variables,  player  actions,  and  variable  impacts  and  how  they 
are  related  to  each  other  and  my  instructional  objectives. 

The  scoring  and  indeed  much  of  the  game  itself  is  based  on  five  variables:  time,  business 
knowledge,  quality,  client  satisfaction,  and  project  status.  The  player’s  final  score  is  determined  at  the 
conclusion  of  the  game  (when  time  has  run  out  or  the  project  has  been  completed)  by  adding  the  player’s 
scores  from  the  time,  quality,  and  client  satisfaction  variables.  A player  has  a maximum  of  50  weeks  to 
complete  the  project  (achieve  100%  in  project  status  variable).  The  time  variable  measures  how  many 
weeks  (turns)  have  been  used.  This  variable  is  impacted  by  player  actions  (which  use  up  one  or  more  week, 
depending  on  the  action),  as  well  as  random  impacts  which  can  add  to  or  reduce  the  number  of  remaining 
weeks  the  player  has.  At  any  time,  the  player  can  look  at  the  screen  and  see  how  many  weeks  he  has  left. 
The  pressure  of  this  time  limit  helps  reflect  to  the  students  the  actual  pressures  that  come  on  a systems 
analyst  in  a real-world  project.  Furthermore,  the  pressure  is  there  to  try  and  help  demonstrate  some  of  the 
conceptual  issues  I cover  in  class,  including  how  the  developer’s  natural  inclination  is  that  taking  time  up 
front  to  document  the  analysis  and  design  of  a system  before  building  it  is  a waste  of  time,  while,  in  reality, 
jumping  into  building  a system  without  establishing  what  the  system  needs  to  do  and  how  these  objectives 
can  be  best  be  met  can  result  in  longer  development  times.  Therefore,  players  are  trying  to  complete  the 
project  in  the  shortest  amount  of  time,  but  also  with  the  highest  quality  and  client  satisfaction. 

Business  knowledge  is  a percentage  rating  that  identifies  the  analyst’s  understanding  of  the  system 
and  the  business  problem  it  is  trying  to  solve.  It  is  heavily  influenced  by  how  much  the  player  has  involved 
stakeholders  in  the  development  process.  This  supports  the  user-centered  design  process  which  I teach  in 
class.  This  score  is  never  revealed  to  the  player,  but  is  used  to  measure  the  impact  of  various  activities 
throughout  the  process.  It  has  a direct  impact  on  the  success  or  failure  of  the  different  activities.  The  state 
of  the  business  knowledge  is  alluded  to  through  conversations  with  stakeholders  but  never  quantifiably 
defined  to  the  player. 

The  quality  variable  is  related  to  the  quality  of  the  system  that  is  being  developed  by  the  player- 
analyst.  This  variable  is  impacted  by  player  actions  and  is  also  heavily  impacted  by  the  business  knowledge 
variable.  This  reflects  the  concept  that  if  an  analyst  does  not  truly  understand  the  business  logic  behind  the 
system,  then  quality  will  likely  suffer.  The  quality  score  can  only  be  determined  through  the  user  testing 
action,  which  will  reveal  what  level  of  quality  the  currently  developed  aspects  of  the  system  are  showing. 

Client  satisfaction  is  a very  important  variable  in  terms  of  player  score.  This  reflects  the  concept 
we  discuss  in  class  that  clients  must  be  involved  or  they  will  have  no  investment  in  the  resulting  system, 
even  if  it  is  a good  one.  Client  satisfaction  is  therefore  impacted  by  the  level  of  stakeholder  involvement  in 
addition  to  the  quality  of  the  system  that  is  developed.  The  level  of  client  satisfaction  is  only  revealed  to  the 
player  by  talking  with  stakeholders  or  conducting  user  testing  on  the  system. 

The  final  variable,  project  status,  simply  shows  what  percentage  of  the  project  has  been 
completed.  This  variable  is  only  impacted  by  the  player  action  implement  system.  To  complete  the  game, 
the  player  must  have  finished  implementing  (building)  the  system  (100%  score)  within  the  50  week  time 
limit.  The  concept  that  an  analyst  can  build  a system  more  quickly  if  he  has  a true  understanding  of  what 
the  system’ s requirements  are  is  reflected  by  the  variable  impact  and  time  taken  by  the  implement  system 
action.  The  higher  the  business  knowledge  score,  the  higher  the  gain  in  project  status  when  implementing 
the  system,  and  the  less  amount  of  time  used  up  by  the  action.  So,  if  the  player  tries  to  implement  the 
system  at  the  very  beginning  of  the  game  without  taking  the  time  to  understand  what  must  be  built  before 
trying  to  build  it,  the  action  takes  a long  time  in  terms  of  weeks  and  results  in  little  gain  in  terms  of 
completing  the  project.  Furthermore,  the  low  business  knowledge  rating  in  this  instance  would  also  result  in 
poor  quality.  This  reflects  how  these  variables  tie  together  to  support  the  underlying  concepts  which  I want 
the  game  to  reinforce. 

Player  actions 

Player  actions  are  the  actions  the  player  can  take  each  turn  of  the  game.  Initially,  there  are  certain 
actions  which  the  player  can  take  and  additional  actions  are  revealed  to  the  player  when  certain  conditions 
are  met,  such  as  talking  with  specific  stakeholders  or  completing  other  actions.  The  following  outline 
shows  the  player  actions  available  in  the  game  and  which  ones  are  initially  available  as  opposed  to  which 
are  “unlocked”  by  other  actions  in  the  game. 

A.  Initially  Available: 

a.  Talk  to  Stakeholders 

b.  Generate/Refine  Requirements  Specification  Document 
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c.  Brush  up  technical  skills 

d.  Take  Ms.  Manner’s  Composition  and  Elocution  Course 

e.  Schedule  Team  Meeting 

f.  Implement  System 

B.  Unlocked  by  Stakeholders: 

a.  Develop  Change  Management  Plan 

C.  Unlocked  by  Actions 

a.  Conduct  Feasibility  Analysis:  Unlocked  by  Generating  Requirements  Specification 
Document. 

b.  Develop/Refine  Use  Cases 

c.  Develop/Refine  Class  Diagrams 

d.  Develop/Refine  Activity  Diagrams 

e.  Develop/Refine  Sequence  or  Communication  Diagrams 

f.  Iterate  Documents 

D.  Unlocked  by  Actions  AND  Stakeholders 

a.  Conduct  User  Test 

The  actions  take  a different  number  of  weeks  to  complete,  with  most  taking  one  week,  some  more, 
and  one-  implement  system  taking  a variable  amount  of  time  based  on  the  player’s  current  business 
knowledge.  Each  action  results  in  different  impacts  on  the  variables  discussed  above.  For  example,  if  a 
player  were  to  talk  to  a stakeholder  as  an  action,  it  would  use  one  of  the  fifty  weeks  available  to  complete 
the  project.  Depending  on  which  stakeholder  the  player  tries  to  talk  to,  the  action  could  unlock  additional 
actions,  unlock  additional  stakeholders,  positively  impact  business  knowledge  and  client  satisfaction,  and 
so  forth.  If  the  player  tries  to  talk  to  certain  stakeholders  without  first  talking  to  the  “boss”  and  getting 
management  support,  he  might  waste  a week  trying  to  talk  with  the  stakeholder  without  getting  any 
response.  This  is  an  example  of  another  concept  we  cover  in  class,  the  need  for  management  buy-in.  As 
another  example,  talking  with  the  secretary  could  also  reveal  the  general  client  satisfaction  level  around  the 
office. 

Other  actions  are  specifically  tied  to  diagrams  taught  and  used  by  the  students  in  class.  The  use 
cases,  class,  activity,  sequence  and  communication  diagrams  are  all  diagrams  the  students  complete  in 
class.  The  students  are  able  to  have  the  sequence  of  these  diagrams,  their  names,  and  what  they  are  used  for 
reinforced  by  playing  the  game.  Each  of  these  actions  help  to  enhance  the  analyst’s  business  knowledge 
score,  and  once  all  of  the  documents  have  been  completed  once,  the  player  is  able  to  iterate  the  documents, 
which  allows  for  them  all  to  be  refined  at  a lower  cost  in  terms  of  time.  This  reinforces  the  concept  of  an 
iterative  system  lifecycle,  as  taught  in  class. 

Finally,  there  are  some  actions  tied  to  additional  concepts  covered  by  the  random  impacts  built  into  the 
game.  I talk  with  the  students  in  class  about  what  skills  are  important  for  a systems  analyst  to  have.  These 
include  strong  communication,  technical,  and  teamwork  skills.  These  concepts  are  reflected  by  negative 
random  impacts  that  can  occur  if  the  player  does  not  have  the  foresight  to  strengthen  each  of  these  areas  by 
using  the  appropriate  action.  This  can  be  further  described  by  looking  at  the  random  impacts  in  the  game. 

Random  impacts 

Random  impacts  are  used  in  the  game  to  both  add  variability  and  increase  the  value  of  replaying 
the  game,  as  well  as  to  reinforce  additional  concepts,  such  as  the  importance  of  various  skills  to  a systems 
analyst.  The  random  impacts  in  the  game  are  as  follows: 

1.  Technical  miscommunication 

2.  Foot-in-mouth 

3.  Poorly  written  communication 

4.  Super  programmer 

5.  Programming  mishap 

6.  Change  request 

7.  Team  argument 

8.  Stakeholder  unavailable 

9.  Team  member  sick 

Each  turn  there  is  a percentage  chance  that  a random  impact  will  occur.  Most  of  the  impacts  are 
negative  and  can  cause  the  player  to  lose  turns,  such  as  if  there  is  an  argument  on  the  team.  The  chance  of 
these  impacts  can  be  reduced  by  certain  actions.  For  example,  the  more  team  meetings  are  scheduled,  the 
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smaller  the  chance  that  if  a random  impact  occurs,  and  that  impact  is  a team  argument,  that  the  negative 
impact  will  actually  occur.  Instead,  the  player  will  be  given  feedback  that  identifies  how  the  team  was  able 
to  handle  a potential  argument  well  and  kept  working  seamlessly.  This  reinforces  the  importance  of 
teamwork  skills.  Other  impacts  are  similar,  such  as  the  change  request.  If  the  player  has  developed  a 
change  management  document,  then  weeks  can  be  lost  adding  additional  functionality  requested  by  the 
client.  The  importance  of  managing  change  requests  is  another  concept  we  discuss  in  class.  One  of  the 
impacts  is  positive,  the  super  programmer.  If  the  player  is  implementing  that  turn  and  this  random  impact 
occurs,  one  of  the  analyst’s  team  members  turns  in  a great  programming  performance,  and  completes  the 
implementation  in  much  less  time  than  normal.  These  impacts  are  meant  to  add  to  the  game’s  fun  and 
unpredictable  nature  and  make  each  replay  different  than  the  previous  while  also  reinforcing  additional 
concepts  covered  in  class. 

Lessons  learned 

My  goals  for  this  project  were  to  create  a fun,  instructional  game  which  could  be  developed  by  one 
undergraduate  student  in  one  semester.  I believe  I have  succeeded  in  this  and  have  also  learned  a number  of 
lessons  along  the  way.  While  I was  able  to  have  the  game  completed  in  one  semester,  the  student  did  not 
manage  to  get  out  all  of  the  bugs,  and  despite  pledging  to  tighten  them  up  over  the  summer,  promptly 
disappeared  and  stopped  answering  emails,  so  a large  lesson  coming  out  of  this  was  to  not  turn  in  that  final 
grade  until  the  game  was  truly  finalized.  While  I can  look  back  on  this  with  a bit  of  a chuckle  now,  I was 
quite  panicked  at  the  time  because  it  was  very  important  that  the  game  accurately  followed  the  underlying 
rules  which  I had  defined  and  the  student’ s final  version  did  not.  I have  been  able  to  locate  a second  student 
now  who  is  removing  these  bugs  and  adding  additional  multimedia  elements,  such  as  sound,  to  the  game, 
so  I believe  my  delay  in  introducing  the  game  to  my  class  will  be  worth  the  wait,  as  the  new  version  should 
be  more  entertaining  and  require  less  reading  on  the  player’s  part. 

Another  important  lesson  which  I learned  in  this  process  is  how  important  it  is  to  come  to  an 
understanding  of  what  the  underlying  rule  structure  of  your  game  will  be  and  how  this  reflects  your 
instructional  goals.  It  was  very  important  for  me  to  design  the  game  and  be  able  to  test  it  on  paper  before 
turning  it  over  to  a developer.  This  allowed  me  to  test  the  logic  that  I had  developed  and  ensure  that  it  was 
sound. 

Ultimately,  while  the  process  of  developing  Lifecycle  was  very  valuable  to  me,  I have  not  yet 
determined  if  the  game  is  a true  success  because  I have  not  yet  turned  it  over  to  my  students.  Once  the 
game  has  been  revised,  and  I reach  the  point  in  the  semester  where  my  students  have  been  introduced  to  all 
of  the  UML  documents  covered  in  the  game,  I hope  to  be  able  to  test  its  impact  on  students  who  play  it.  I 
envision  this  game  being  incorporated  into  the  class  as  a form  of  scaffolding.  I believe  it  will  be  very 
important  to  debrief  the  students  and  talk  about  lessons  learned  from  their  experience.  Students  must  be 
asked  to  analyze  their  play  and  what  they  learned  from  the  game  or  it  will  likely  remain  nothing  more  than 
a game  to  them.  I think  this  analysis  both  in  a group  discussion  and  possibly  through  a journal  assignment, 
would  be  very  important  to  successfully  using  an  instructional  game  like  Lifecycle.  In  conclusion,  this  was 
a challenging  but  valuable  experience  which  I hope  will  prove  valuable  to  others.  Much  research  remains 
before  I can  call  Lifecycle  a success,  but  the  first  steps  have  been  taken,  and  it  is  my  hope  that  this  will  help 
others  to  begin  their  own  journey  into  designing  instructional  games. 
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The  computer  literacy  class  - back  to  basics 

David  Winograd 
York  College 

Abstract 

Computer  literacy  is  a phrase  that  is  often  quite  elusive.  How  does  it  relate  to  information  literacy?  Is  there 
a body  of  information  skills  that  comprise  computer  literacy  for  everyone,  and  if  not,  can  it  be  called 
‘ literacy  ’?  How  is  it  implemented  and  assessed?  Answers  vary  according  to  what  program  you  view  and 
the  assumptions  behind  the  programs.  This  paper  surveys  assessment  and  administration  of  a number  of 
computer  literacy  programs  and  literature  and  extrapolates  its  findings  onto  the  case  of  a computer 
literacy  program  of  an  urban  east  coast  college 


What  is  computer  literacy? 

Definitions  of  computer  literacy  vary  according  to  the  type  of  academic  program  in  which  the 
learned  skills  will  be  used  (N.D.,  2005;  Zestotarski,  2000).  In  common  discussion,  computer  literacy  means 
little  more  than  the  ability  to  use  very  specific  programs  such  as  Microsoft  Word,  Internet  Explorer  and 
Microsoft  Outlook,  and  to  perform  well  defined  tasks  by  rote.  This  is  fine  in  theory,  but  people  use 
computers  to  solve  problems  and  achieve  results.  The  effect  of  this  rote  mode  of  learning  can  be  like  a child 
who  has  memorized  a children’s  book  and  can,  on  demand,  recite  it  back,  but  not  know  the  reason  why 
(N.D.,  2005;  Patricia  Iannuzzi  & Donald  W.  Farmer,  2000;  Perkins,  1992). 

Information  literacy  is  a term  that  often  gets  confused  with  computer  or  technological  literacy. 
Information  literacy  is  a superset  of  computer  literacy.  A 1999  report  from  the  research  Council  suggests 
that  the  concept  of  ‘fluency’  is  quite  important.  Computer  Literacy  is  based  in  rote  learning  of  hardware 
and  software  applications  without  a particular  context.  Shulman,  (1996)  proposes  the  integration  of 
constructivist  techniques  with  the  reasoning  that:  “In  a real  life  situation,  one  needs  the  ability  to  build  and 
act  upon  individual  realities,  while  in  a situation  where  the  learners  are  being  trained  to  think  and  respond 
in  the  exact  same  manner,  such  as  in  military  training,  it  is  counter-productive  to  view  differing  realities. 
Fluency,  with  technology,  is  the  understanding  of  the  underlying  concepts  of  technology  and  as  it  relates  to 
a K-12  classroom,  and  are  concerned  with  a ‘deep  understanding’  of  technology  and  the  eventually 
increasing  use  of  it.  (Patricia  Iannuzzi  & Donald  W.  Farmer,  2000). 

The  need  for  authentic  instruction  is  critical  if  the  rote  learned  skills  are  to  become  part  of  a new 
computer  user’s  repertoire.  Giving  our  students  no  more  than  knowing  what  button  to  push  is  not  doing 
them  a service.  What  is  needed  is  to  give  students  a clear  understanding  that  a computer  is  a tool  and  is  not 
a thing  onto  itself,  but  merely  designed  to  help  solve  problems  and  accomplish  work  (Vaz,  2004).  A basic 
failing  in  trying  to  differentiate  computer  literacy  and  information  literacy,  is  that  nothing  stands  still. 

There  is  always  new  information  to  learn  and  new  concepts  to  learn  how  to  employ.  Just  a short  number  of 
years  ago  there  were  no  USB  Flash  Drives.  This  is  a wonderful  device  that  solves  the  fragility  issue  of 
floppy  disks,  and  sells  much  cheaper  than  any  other  removable  media.  Everyone  storing  data  should  have  a 
flash  drive,  but  not  just  to  own  a new  toy.  The  case  must  be  made  that  it  is  good  workable  technology  that 
can  solve  a salient  problem  and  not  be  one  of  the  many  marketed  solutions  in  search  of  a problem. 
Demonstrations  should  be  done  on  how  to  use,  but  more  importantly,  why  to  use  them.  A case  must  be 
made  to  convince  students  of  this,  or  most  any  other  technologies’  merit,  since  most  people  have  had 
problems  with  floppy  disks  not  being  able  to  be  read  between  machines.  In  class,  we  take  apart  a floppy 
disk  and  the  class  sees  just  how  fragile  it  is.  We  put  a USB  Drive  in  a Mac,  run  something,  put  it  in  a PC, 
run  the  same  file,  toss  it  up  in  the  air  and  let  it  hit  the  ground.  When  we  are  done,  it  goes  back  into  a 
computer  and  it  always  works.  This  has  proved  to  be  a motivating  demonstration,  solving  a problem  and 
building  fluency.  It  seems  that  in  the  strictest  of  definitions,  information  literacy  is  more  to  be  sought  than 
computer  literacy.  As  Computer  Literacy  gives  you  a tour  of  the  tools,  Information  Literacy  is  concerned 
with  how  to  make  the  best  use  of  the  tools. 

As  stated  by  the  American  Library  Association,  four  aspects  were  found  to  comprise  information 

literacy: 

1 . The  ability  to  recognize  when  information  is  needed; 
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2.  The  ability  to  locate  the  needed  information; 

3.  The  ability  to  evaluate  the  suitability  of  retrieved  information,  and 

4.  The  ability  to  use  effectively  and  appropriately  the  needed  information.  (American 
Library  Association,  1989) 

This  is  quite  general,  and  doesn’t  appear  to  address  the  how,  where,  why,  and  when  of  the 
situation.  We  feel  that  these  are  not  objectives,  but  outcomes  that  can  be  met  any  number  of  ways  either 
with  or  without  technology.  More  interesting  is  Vitolo  and  Coulston’s  Information  Literacy  Taxonomy, 
which  states  that:  “With  this  approach  the  disciplines  express  a degree  of  fluency  appropriate  for  their  field. 
Once  the  amount  of  fluency  is  specified,  then  individuals  can  be  trained  to  meet  discipline- specified  needs 
and  not  simply  be  trained”  (Vitolo  & Coulston,  2002).  Mere  knowledge  and  demonstrated  proficiency  in 
the  nuts  and  bolts  of  computer  literacy  may  have  been  enough  in  the  past,  but  as  technology  moves  on,  we 
feel  that  information  literacy,  as  it  relates  to  technology,  is  crucial  to  the  success  of  the  student  and  that 
learning  rote  information  without  context  may  be  quickly  forgotten. 

Administration  of  Computer  Literacy  Competencies 

In  surveying  how  various  institutions  administer  tests  and  courses  to  measure  computer  literacy, 
we  found  a number  of  differing  procedures  and  we  would  like  to  compare  and  contrast  these  procedures 
with  York  College,  an  urban  four-year  undergraduate  university  that  is  part  of  the  City  University  of  New 
York  system. 

Some  institutions  have  contracted  with  third  party  vendors  to  provide  test-out  testing  for  computer 
literacy  skills.  Longwood  University  in  Virginia  is  one  of  these.  It  has  contracted  with  a testing  service 
called  Smartforce.com,  now  called  Skillsoft.com,  (Skillsoft.com,  2005)  to  administer  online  computer 
literacy  testing  which  can  be  done  from  any  Internet  browser  twenty-four  hours  a day.  This  can  be  taken 
any  time  before  graduation.  We  were  not  able  to  see  the  test,  but  did  find  that  a number  of  online 
remediation  books  were  available  from  skillsoft.com.  This  minimalist  orientation  to  the  problem  seems  to 
be  well  outsourced,  but  regardless  of  the  quality  of  the  testing  or  remediation,  the  fact  that  this  must  be 
completed  any  time  before  graduation  appears  to  be  detrimental.  If  a purpose  behind  the  attainment  of 
computer  literacy  was  to  use  the  skills  in  the  course  of  a student’ s undergraduate  tenure,  this  procedure 
seems  counter-productive  at  best. 

At  Cal  State  San  Marcos,  the  Instructional  and  Information  Technology  Services  department  does 
the  testing  of  computer  literacy  in-house.  A good  deal  of  online  information  and  structure  has  been 
provided  on  their  web  site.  Online,  one  can  find  a sample  test  to  be  used  for  practice,  a number  of  articles 
and  glossaries  explaining  the  basics  of  computer  literacy  and  a listing  of  many  Colleges  and  Universities 
along  with  the  course  numbers  of  computer  literacy  classes  the  listed  institutions  provide  that  will  act  as 
waivers  of  the  computer  literacy  test  for  transfer  students.  According  to  Cal  State,  the  test,  if  the 
requirement  is  not  met  otherwise,  must  be  taken  during  the  first  or  second  semester  of  enrollment,  which 
seems  advisable.  The  sample  test  however  appears  subjective  and  quite  hard  to  assess.  The  answers  to 
questions  like:  “What  is  the  Dow  Jones  Average?”  or  “What  is  the  temperature  in  New  York  City?”  change 
minute  to  minute,  and  unless  the  computer  instantly  scores  itself  the  same  second  the  answer  is  given,  the 
answer  would  vary.  The  test  may  be  taken  as  many  times  as  a student  needs  to  attain  a passing  grade.  An 
exception  to  most  other  programs  is  that  no  courses  are  offered  to  teach  computer  literacy.  Instead  students 
are  directed  to  certain  books  held  in  reserve  in  the  library  or  advised  to  call  for  tutoring.  This  may  not  be 
adequate  for  all  students  and  it  seems  like  a piece  of  a well-designed  puzzle  is  missing. 

During  the  orientation  week  for  freshman  at  SUNY  Brockport,  students  are  invited  to  take  a 
computer  literacy  test  that,  if  passed,  will  meet  all  school  requirements  for  technology.  A sample  test  is 
online  and  the  skills  are  quite  basic.  The  sample  test  requires  both  working  on  a simulated  Microsoft  Word 
document  with  a button  which  when  pushed,  displays  the  correct  answer  or  form,  along  with  a multiple 
choice  section  that  scores  itself  (SUNY  Brockport,  2004).  We  seriously  doubt  if  this  amount  of 
information  is  adequate  for  many  majors  and  hope  that  the  instruction  is  furthered  within  class  work. 

At  St.  Edwards  University  in  Texas,  there  is  a six-part  test  based  in  Internet  communications, 
library  research,  word  processing,  spreadsheets  and  presentations  that  must  be  taken  prior  to  graduation.  As 
previously  noted  we  consider  late  attainment  of  skills  a problem  and  this  is  verified  by  St.  Edwards  by 
posting  that  although  the  test  is  a graduation  requirement,  it  is  suggested  that  the  test  be  completed  early  in 
a student’s  academic  career  since  most  of  the  skills  will  be  used  in  many  classes  and  that  the  classes  will 
not  teach  the  skills.  A student  can  elect  to  take  any  of  the  modules  together  or  separately,  can  take  the  test 
as  many  times  as  needed  for  successful  completion,  or  take  a six  week  no  credit  remedial  course  that  will 
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prepare  them  for  the  test.  A downside  of  this  and  similar  programs  as  we  see  it,  is  that  concepts  are 
assessed  in  no  apparent  context.  There  are  no  courses  that  if  taken  at  other  institutions,  will  allow  one  to 
test-out  of  the  St.  Edwards  proprietary  test. 

The  College  of  Southern  Idaho  has  a similar  approach  but  the  content  is  slightly  different  bringing 
up  a question  of  what  a student  needs  to  know.  If  you  attend  one  school  the  set  of  skills  is  different  from  the 
set  taught  and  assessed  at  another  school.  In  this  College,  students  need  to  take  a four  pronged  test 
assessing  competency  in  general  technology,  Internet,  email,  word  processing  and  a choice  of  either 
spreadsheet  or  presentation  applications.  Again  this  may  be  taken  as  many  times  as  needed.  If  the 
information  is  not  known,  courses  and  self-study  aids  are  available.  Once  again,  this  is  a graduation 
requirement.  The  base  of  knowledge  used  since  spring  of  2005  has  been  that  of  the  Internet  and  Computing 
Core  Certification  (IC3)  created  by  Certiport  Inc.  (Certiport  Inc.,  2005),  although  they  do  not  use  the 
Certiport  test.  This  test  uses  Macromedia  Flash  simulations  and  encompasses  contextual  information  as 
well  as  basic  computer  literacy  concepts.  Along  with  Certipoint  and  Skillsoft.com  there  appears  to  be  a 
movement  toward  standardized  testing  of  computer  literacy  skills  and  the  integration  of  authentic 
contextualization  of  skills  into  problem  solving  environments.  The  Educational  Testing  Service  announced 
at  the  end  of  2004  that  they  planned  to  unveil  a test  to  assess  both  computer  and  information  literacy  skills 
as  they  relate  to  computer  based  technology  (Young,  2004).  Based  on  the  advice  of  representatives  of  seven 
Universities,  the  idea  behind  the  test  acknowledges  that  most  students  arrive  at  school  with  certain 
computer  skills  such  as  downloading  music  or  video  files  and  using  chat,  but  these  are  not  the  skills  needed 
to  do  well  at  school.  Schools  prize  the  ability  to  research  and  evaluate  information  from  multiple  sources. 
This  new  test  purportedly  brings  information  and  computer  literacy  together  and  in  announcing  the  test,  the 
ETS  proposed  that  the  test  was  not  just  for  basic  computer  literacy,  but  for  problem  solving  and 
contextualization  of  skills  as  well.  Recently  the  ETS  has  announced  a version  of  this  test  for  high  school 
students  to  assist  in  assessing  and  meeting  computer  literacy  requirements  in  higher  education  (Young, 
2005).  If  this  proves  successful  at  a high  school  level,  it  may  be  the  creation  of  a new  SAT  type  test  and  a 
sizable  revenue  stream  for  ETS.  Although  still  in  testing,  it  does  raise  the  problem  of  specific  contexts  of 
technology  usage  (Cornish,  2005).  Someone  majoring  in  physics  would  use  Microsoft  Excel  differently 
than  someone  majoring  in  business.  Similarly,  graphics  arts  majors  will  use  Adobe  Photoshop  differently 
than  would  someone  majoring  in  secondary  education.  Both  programs  have  massive  feature  sets  and  not 
many  users,  if  any,  use  every  feature  in  both  programs  and  certainly  not  for  the  same  purpose.  Therefore 
the  ‘one  size  fits  all’  operating  concept  of  standardized  testing  may  not  align  well  with  computer  literacy. 

One  solution  to  this  potential  problem  is  to  follow  the  direction  of  schools  such  as  Potsdam 
College  and  Western  Illinois  University.  At  Potsdam,  the  basic  computer  literacy  course  was  phased  out 
and  in  its  place  the  skills  required  were  determined  and  subsequently  incorporated  into  coursework  in  a 
student’s  major  area  of  study.  At  Western  Illinois  University  a committee  of  the  faculty  senate  determined 
that  an  integrated  program  aimed  at  developing  computer  competencies  should  be  a shared  responsibility  of 
every  department  of  the  university.  General  education  courses  along  with  other  lower  level  courses  should 
incorporate  appropriate  technology.  Two  sets  of  competencies  would  be  necessary  for  graduation.  The 
standard  college-level  competency  test  must  be  passed  before  the  end  of  the  second  year  of  enrollment  and 
the  professional  competency  test  must  be  passed  before  graduation.  The  specifics  of  the  professional 
competency  test  are  left  up  to  each  academic  department  to  deploy  as  it  sees  fit  (Western  Illinois 
University,  2000). 

We  feel  that  basic  computer  literacy  should  be  a responsibility  of  the  entire  campus  with 
academics  departments  working  with  the  IT  or  computer  services  department  to  develop  a plan  for  each 
academic  major  concentration.  This  is  probably  easier  than  it  sounds  since,  at  least  in  Potsdam;  the  teacher 
education  department  outsourced  the  basic  computer  literacy  course  to  the  department  of  instructional 
communication  and  technology  for  many  years.  It  also  becomes  a problem  when  people  of  various  abilities 
and  backgrounds  were  enrolled.  One  time  when  we  held  such  a class,  we  were  asked  two  questions  the  first 
day  of  class.  The  first  question  was  asking  if  we  would  learn  to  code  JavaScript?  The  second  question 
asked  what  was  a word  processor?  As  should  be  apparent,  it  is  not  a trivial  task  to  find  a level  for  the  class 
that  won’t  bore  the  more  advanced  or  lose  the  beginners. 

The  Challenges  and  Potential  Solutions  for  York  College 

Challenges 

The  computer  literacy  requirement  at  York  is  required  of  every  student  who  enrolls.  This  is  a 
fourteen- week,  one-credit  course  meeting  I-V2  hours  a week.  Homework  must  be  given  the  majority  of 
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weeks  and  there  are  quite  a few  class  projects,  causing  animosity,  since  the  general  feeling  of  the  students 
is  that  this  is  too  much  work  for  a one-credit  class.  Since  everyone  who  does  not  test  out  needs  to  go 
through  the  class  before  graduation,  it  is  quite  common  for  people  to  pick  up  the  last  credit  on  the  way  out 
the  door,  thereby  eliminating  any  possibility  of  using  the  instruction  for  their  work  at  York. 
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Table  2. 


Computer  Literacy  Course  by  Enrolled 
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Tables  1 and  2 make  the  point  quite  well.  In  2004  only  about  12%  took  the  course  during  their  first  year 
while  the  majority  took  it  during  their  third  year.  This  has  wasted  two;  maybe  three  years  where  students  could  be 
making  use  of  learned  skills.  In  both  calendar  years  the  fourth  academic  year  had  considerable  representation.  Since 
York  can  be  completed  in  four  or  five  years,  the  large  number  of  fourth  year  students  taking  the  course  is  akin  to 
getting  ones  last  one  credit  ticket  punched  before  graduation.  The  general  feeling  of  students  taking  the  course  at 
this  late  date  is  that  they  don’t  want  to  be  in  the  class  and  will  do  minimum  work 

Another  possible  pitfall  is  teacher  expectations  of  students  in  areas  of  the  student’s  majors  were  not  aligned 
with  what  was  being  taught  in  the  computer  literacy  course.  We  polled  three  upper-class  sections  of  a course  dealing 
with  multimedia  integration  into  curriculum  for  teachers.  They  were  asked  to  answer  what  technology  expectations 
teachers  in  their  major  subjects  expected  them  to  enter  class  possessing.  This  ranged  over  a number  of  majors  but 
all  students  were  in  Teacher  Education. 
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Table  3. 


Major  Subject  Teacher  Expectations 


% of  Students  Reporting 


Table  3 shows  that  after  only  30%  of  teachers  expecting  knowledge  of  Microsoft  Word,  the  next  largest 
expectation  was  that  students  possess  no  computer  knowledge.  We  left  items  off  the  chart  that  received  one  percent 
or  under.  These  items  were:  databases,  search  engines,  web  site  evaluation,  computer  basics  and  graphic 
manipulation.  The  survey  suggests  that  students  are  being  taught  competencies  that  will  be  unnecessary.  It  may  also 
suggest  that  some  instructors  are  not  comfortable  enough  with  technology  to  integrate  its  use  into  their  classes.  This 
appears  to  be  a major  disconnect. 

The  basic  computer  literacy  course  had  a bad  reputation  according  to  many  students.  It  was  developed  in 
part  and  administered  by  an  individual  with  an  engineering  background  and  seemed  to  be  too  technical  and 
computer  science  oriented  for  its  student  base.  According  to  the  homework  manual,  the  subjects  seemed  to  be 
presented  without  much  discernable  context.  It  was  common  to  hear  students  complain  that  they  had  a harder  time 
understanding  the  assignment  than  performing  it. 

We  feel,  as  recommended  by  the  tenets  of  Instructional  Design,  that  context  is  critical  and  without  context 
there  can  be  no  long-term  transfer  (Dick  et  al. , 2001).  The  course  was  later  changed,  tossing  out  the  old  materials 
and  keeping  the  idea  of  context  in  the  forefront,  creating  a curriculum  in  which,  taking  a cue  from  John  Keller, 
(1987)  relied  heavily  on  the  relevance  of  the  concepts  to  individual  students.  Keller  stated  that  to  achieve  motivation 
there  must  be  four  conditions:  attention,  relevance,  comfort  and  satisfaction.  Unless  relevancy  is  obtained  the 
instruction  can  be  nothing  more  than  disconnected  data,  easy  to  forget.  The  mode  most  often  used  in  the 
restructuring  of  the  course  was  to  have  the  students  read  about  non-contextualized  concepts  before  the  class  and  then 
provide  context  and  relevancy  within  classroom  discussion.  To  date  these  and  ongoing  changes  have  improved 
student  perception  of  the  course  and  resulted  in  a higher  percentage  of  homework  submitted. 

The  problem  of  testing  out 

Since  the  basic  computer  literacy  course  is,  as  stated,  quite  basic,  there  are  many  people  who  do  not  need 
this  course  but  take  it  regardless  since  it  is  a graduation  requirement  or  they  need  the  one  credit  it  provides.  A test 
was  developed  to  allow  people  to  test-out  of  the  course.  This  competency  test  went  through  a number  of  iterations 
until  we  were  confident  that  it  provided  a realistic  assessment  of  the  competencies  acquired  in  the  computer  literacy 
course.  To  our  dismay,  it  was  made  clear  that  York  was  looking  for  a competency  test  that  would  assess  not  only 
the  skills  in  the  computer  literacy  course,  but  also  all  the  skills  of  a taxonomy  of  computer  competencies  that  was 


543 


developed  some  years  back  by  the  Academic  Communications  and  Educational  Technology  department  in 
conjunction  with  other  disciplines  on  campus.  In  general  these  competencies  include  the  following  topics:  operating 
system,  word  processing,  e-mail  usage,  internet  browsing,  spreadsheets,  graphic  usage  and  presentation  tools.  To  be 
added  were  concepts  and  terminology.  Each  heading  was  broken  down  into  3 to  1 1 base  elements.  The  computer 
literacy  course  according  to  the  taxonomy  was  meant  to  meet  the  operating  system,  word  processing  and  email 
usage  requirements  only,  it  met  more  than  that,  including  concepts  and  terminology,  which  provide  context  and 
graphic  manipulation  as  a function  of  word  processing  among  others,  but  it  certainly  fell  short  of  covering  the  entire 
taxonomy.  The  missing  competencies  were  meant  to  be  addressed  by  more  advanced  courses.  So,  following  York’s 
position  to  test  out  of  a basic  computer  literacy  class,  one  is  required  to  be  competent  in  double  the  amount  of 
capabilities  taught  in  the  literacy  course.  The  effect  of  this  is  to  run  6 sections  of  the  computer  literacy  course  each 
semester  when  in  our  opinion  it  could  be  well  covered  by  3 or  4 sections. 


Potential  Solutions 

In  viewing  various  ways  that  institutions  deal  with  this  and  other  problems  of  computer  literacy,  it  has 
become  apparent  that  computer  literacy  is  a term  that  is  ready  to  be  put  to  rest.  Without  appropriate  context  the 
potential  for  learning  is  greatly  diminished.  Additionally,  there  is  a real  question  as  to  exactly  what  comprises 
computer  literacy.  The  word  ‘literacy’  according  to  Brian  Harvey,  a computer  scientist  at  Berkeley  wrote  in  1983 
that  computer. . .literacy  means  that  everyone  must  have  the  experience  in  order  to  function  at  all  in  our  society 
(Harvey,  1983),  and  then  goes  to  question  what  those  skills  may  be  and  if  they  are  the  same  for  everyone?  Given  this 
environment  of  conflicting  ideas  over  computer  literacy  as  a thing  onto  itself  and  how  we  feel  that  it  should  be 
broadened  at  all  levels  to  become  a supported  and  well  administered  part  of  information  literacy,  instead  of  working 
at  cross  purposes,  we  propose  the  following  potential  solutions  to  solve  the  problems  at  York  College  and  elsewhere. 

1.  Integration  of  specific  uses  of  technology  in  contextually  appropriate  subject  areas  for  each  major. 

2.  Hiring  teachers  within  academic  departments  that  are  capable  of  providing  such  instruction. 

3.  Development  of  individual  departmental  technology  requirements  appropriate  to  each  field. 

4.  Developing  sessions  during  freshman  orientation  week  where  incoming  students  will  be  encouraged  to  test 
out  on  a departmental  level.  These  test-out-tests  will  be  developed  in  conjunction  between  the  IT 
department  and  the  academic  units. 

5.  Requiring  that  basic  computer  proficiency  be  attained  by  no  later  than  the  end  of  the  second  semester 
before  allowing  continuation  into  sophomore  year.  This  would  require  a change  in  the  advisement 
procedure. 

6.  Developing  two  part  remediation  courses  where  the  first  part  will  be  general  computer  basics  and  the 
second  half  will  be  handled  by  individual  academic  departments. 

7.  The  creation  of  a series  of  test-out  tests  instead  of  one  all  encompassing  test.  We  suggest  that  there  be  one 
test-out  test  for  the  computer  literacy  course  and  then  a series  of  departmentally  administered  tests  given  as 
prerequisites  for  further  course  work.  The  content  of  these  tests  and  required  competencies  would  vary 
according  to  the  needs  of  the  academic  department. 

8.  Publishing  the  set  of  skills  that  will  be  tested  to  allow  students  to  concentrate  on  the  proper  information. 

9.  Raising  the  number  of  credits  or  lowering  the  student  workload  for  the  computer  literacy  course. 

10.  Offering  ongoing  consultation  and  tutoring  to  allow  students  with  absolutely  no  computer  or  typing 
experience  to  be  on  a level  playing  field  with  the  average  student. 

The  idea  of  one  department  deciding  what  knowledge  will  be  privileged  for  all  departments  may  do 
disservice  if  the  students  do  not  learn  what  their  major  academic  department  thinks  they  should  know,  or  if  they 
are  taught  information  one  way  and  according  to  the  pedagogy  of  their  major,  need  to  unlearn  skills  and  learn  it 
again  in  differing  contexts.  A partnership  must  be  developed  between  IT  and  the  academic  departments  who  are 
served  by  them  for  the  betterment  of  the  students.  When  it  comes  to  computer  literacy,  one  size  does  not  fit  all. 
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Towards  the  Design  and  Development  of  Individualized  Online  Instruction  in 

Higher  Education 

Ying  Xie 

Penn  State  University 

Abstract 

The  development  of  computer  technologies  promised  highly  for  realizing  individualized  instruction  online. 
This  paper  reviews  and  reflects  on  research  efforts  and  practices  that  have  been  made  toward  the  design  and 
development  of  individualized  online  instructions  in  higher  education.  Based  on  the  comparative  analyses  of  those 
studies  with  two  traditional  distance  education  design  theories,  key  principles  are  recommended.  Such  review  and 
reflection  are  of  primary  importance  for  researchers  and  practitioners  in  furthering  their  explorations  and 
practices. 


Introduction 

The  advent  and  development  of  computer  and  telecommunication  technologies  awakened  many  researchers 
and  educators  with  the  dream  of  realizing  individualized  instruction.  However,  while  a decade  went  by  since  the 
wide  spread  use  of  computer  and  computer-associated  technologies,  little  was  achieved  toward  practical  and 
successful  design  and  development  of  individualized  instruction. 

“Individualized  instruction”  and  “personalized  learning”  were  used  interchangeably  in  the  literature.  In 
fact,  what  instruction  is  to  learning  is  what  a means  to  an  end.  That  is,  learning  is  a possible  end  to  instruction.  What 
an  instructor  and  instructional  designer  can  do  is  to  carefully  design  and  implement  the  instruction  so  that  learning 
may  be  maximized.  It  is  generally  believed  that  when  instructions  are  customized  to  a specific  individual’s  needs, 
maximum  learning  can  occur.  This  paper  aims  to  present  a review  of  recent  literature  that  addressed  the  issue  of  how 
such  instruction  should  be  or  were  designed  and  developed. 

Recent  Research  On  Learner  Differences 

There  has  be  an  identifiable  and  continuing  effort  in  the  literature  that  focused  on  finding  the  relationship 
between  students’  learning  styles  and  their  success  in  and  satisfaction  with  distance  learning  (Dillon  & Greene, 

2004;  Moore,  2002).  Bollet  & Fallon  (2002)  defined  learner  as  “our  own  individual  internal  way  of  processing 
information  via  our  brain’s  preferred  learning  style”.  It  was  supported  by  many  authors  and  researchers  that  learning 
styles  decided  how  students  learn  (for  example,  Carnwell,  2000;  Thompson,  1998).  They  believed  that  students’ 
achievement  and  satisfaction  must  be  a function  of  and  could  be  predicted  by  their  learning  styles.  Therefore,  in 
order  to  enrich  students’  learning  experiences  and  to  personalize  their  own  e-learning  processes,  “the  diversity  of 
learning  styles  must  be  in  the  conscious  awareness  of  the  instructor/trainer  at  all  times”  (Bollet  & Fallon,  2002.  p43). 
As  Dillon  & Greene  (2004)  mentioned,  “the  recognition  that  learners  have  different  needs  is  indeed  a revolutionary 
theory,  one  that  promised  to  move  us  from  mass  education  to  individualized  learning”  (p9).  A number  of  studies 
were  conducted  along  this  line  of  inquiry  with  the  hope  that  suggestions  could  be  made  for  adjusting  instructions 
based  on  the  findings  of  significant  effect  of  learner  differences  on  their  academic  achievement. 

Dillon  & Greene  (2004)  performed  a pretty  exhaustive  review  on  the  studies  about  learning  differences  and 
online  learning  environment.  The  studies  they  reviewed  were  most  done  in  the  90’ s.  This  paper  summarizes  the 
studies  conducted  onwards  in  the  same  manner. 

Learning  Styles  And  Achievement 

Neuhauser  (2002)  found  no  effect  of  learning  modality  preferences  (visual,  auditory,  and 
kinesthetic/tactile)  and  temperament  preferences  (introversion/extraversion,  sensation/perceiving,  sensation/judging, 
intuition/thinking,  and  intuition/feeling)  on  students’  achievement  in  an  online  course. 

In  another  study  (DeTure,  2004),  the  hypothesis  that  cognitive  style  scores  could  predict  student  success  (in 
terms  of  GPA)  in  an  online  education  course  was  rejected.  The  cognitive  styles  in  this  study  included  field 
dependence/independence.  In  addition,  the  second  hypothesis  in  the  same  study  was  also  rejected  which  stated  self- 
efficacy  could  predict  student  success. 
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Learning  Styles  And  Satisfaction 

Kanuka  & Nocente  (2003)  found  in  their  study  that  students  demonstrated  different  personality  types  as 
measured  by  the  Millon  Index  of  Personality  Styles  (MIPS).  This  instrument  measured  three  aspects  of  personality 
types  including  learners’  motivating  aims,  cognitive  modes,  and  interpersonal  behavior.  However,  these  differences 
in  their  personality  characteristics  failed  to  influence  their  satisfaction  on  the  web-based  instruction.  The  majority  of 
the  participants  were  satisfied  with  the  course  regardless  of  their  personality  differences. 

Conclusion 

Not  surprisingly,  the  findings  in  the  reviewed  research  studies  were  consistent  with  one  another  in  that 
leaner  differences  were  found  no  or  little  effect  on  academic  success,  satisfaction,  persistence,  completion  of  course 
(Dillon  & Greene,  2004).  In  response  to  those  insignificant  results,  Dillon  & Greene  eventually  abandoned  the 
fundamental  idea  of  realizing  individualized  instruction  by  catering  to  different  learner  needs.  Both  authors 
continued  to  propose  that,  “rather  than  focusing  upon  how  to  modify  the  instruction  to  accommodate  the  needs  of 
the  learner,  we  should  instead  focus  upon  modifying  the  learning  to  accommodate  the  demands  of  the  instruction.” 
(p9).  The  “learning”  in  their  statement  meant  learning  styles  and  strategies  on  learner’s  end.  They  reinforced  their 
assertion  by  quoting  the  goal  of  all  education  as  “to  help  learners  learn  how  to  learn”  (p5).  In  place  of  individualized 
instruction,  they  suggested,  “a  more  powerful  and  expedient  method  of  address  individual  learner  needs  may  be  to 
identify  effective  approaches  to  learning  and  then  help  students  acquire  the  metacognitive  skills  needed  to  adopt 
those  approaches  in  setting  where  they  have  been  found  to  lead  to  success”.  (p9) 

If  a closer  look  is  taken  at  the  studies,  it  is  not  hard  to  find  that  participants  of  these  studies  were  either 
college  students  or  adult  learners.  If  learners  chose  to  study  from  a web-based  instruction  that  was  generally 
regarded  as  rather  isolated  independent  study,  it  is  reasonable  to  assume  that  the  majority  of  such  learners  were 
motivated  to  begin  with.  Even  if  learning  styles  could  have  an  effect  on  their  learning  achievement,  the  “noise”  of 
high  motivation  might  have  diluted  the  effect.  If  this  is  the  case,  to  sustain  and  even  better  enhance  students’ 
motivation  in  an  online  learning  environment  is  crucial  to  make  possible  a high  degree  of  academic  achievement  and 
satisfaction.  In  the  other  words,  to  individualize  online  instructions,  we  must  find  effective  and  adaptive  strategies  to 
motivate  individual  learners. 


Implementation  Examples 

Is  individualized  instruction  just  a dream  that  is  never  attainable?  Even  though  the  findings  regarding  to  the 
effects  of  learner  differences  on  online  learning  were  discouraging,  several  instructional  designs  toward  personalized 
instructions  were  proven  successful  and  received  positive  feedback  from  students.  One  of  the  designed  applied 
learner-centered  method  and  another  employed  multi-model,  metadata  drive  approach  to  create  adaptive 
personalized  e-learning.  These  methods  will  be  depicted  in  detail  as  follows. 

Learner-centered  Instructional  Design  And  Development  (LCD) 

Parchoma  (2003)  summarized  and  compared  two  examples  of  successful  instructional  designs  that  used 
learner-centered  approach  as  opposed  by  domain-centered  design  (Sims,  2001).  Domain-centered  design,  by 
definition,  tends  to  ignore  the  needs  and  characteristics  of  the  learners,  instead  focuses  on  “design  and  development 
activities  that  lead  to  well-organized  and  well-presented  knowledge  objects”  (Parchoma,  2003.  p38).  In  contrast, 
learner-centered  design  focuses  on  “demographic  and  cognitive  profiles  of  learners,  prior  knowledge,  perceptions, 
preferences,  needs,  goals,  characteristics,  and  experiences  of  learners”...  and  on  “ensuring  that  essential  content  is 
contextualized  in  learner  experiences  and/or  goals  so  that  learners  will  be  motivated  to  value  it”  (Parchoma,  2003. 
p38). 

In  practice,  learner-centered  design  was  realized  by  involving  learners  in  all  phases  of  analysis,  design, 
development,  implementation,  and  evaluation  (ADDIE).  Rich  information  was  collected  about  target  learners  in  the 
analysis  phase  and  learner  profiles  were  set  up  to  blueprint  the  course  plan.  Such  information  included, 

“demographic  information,  comfort  with  and  access  to  technology,  learning  styles  and  preferences,  comfort  with 
language  of  instruction,  competing  responsibilities  and  learning  goals,  academic  self-concept,  achievement 
experiences,  perceptions  of  authority,  and  idiosyncrasies”  (Parchoma,  2003.  p40).  Such  learner  profiles  were  used  to 
select  “course  features  and  delivery  modes”  (Parchoma,  2003.  p43).  Target  learners  participated  in  the  task  analysis 
so  that  “the  inclusion  of  options  may  allow  learners  to  choose  personalized  learning  activities  and  pathways” 
(Parchoma,  2003.  p41).  Besides  task  analysis,  students  also  actively  provided  feedback  on  the  effectiveness  of 
scaffolding  and  self-monitoring  strategies  when  the  course  was  being  designed  and  developed. 

At  the  implementation  phase,  instructors  interacted  with  each  individual  student  and  efforts  were  made  to 
establish  instructor-learner  rapport.  To  foster  “trust  and  caring”  in  the  learning  environment,  not  only  the  instructors 
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offered  “timely  support  and  encouragement”  to  each  individual,  but  also  “opportunities  for  off-task  student- to- 
student  communication”  were  provided  so  that  sense  of  isolation  in  learners  could  be  overcome  (Parchoma,  2003. 
p41).  Finally,  deliberation  was  exercised  in  the  evaluation  phase,  “LCD  emphasized  formative  rather  than 
summative,  iterative  rather  than  periodical,  and  lateral  rather  than  hierarchical  evaluation  strategies”  (Parchoma, 
2003.  p41). 

Two  courses  were  designed,  developed  and  piloted  in  two  different  institutions  by  utilizing  the  LCD  design 
principles.  These  courses  turned  out  to  be  a huge  success  on  account  of  the  unprecedented  completion  rate  and 
positive  remarks  given  by  the  students. 

In  conclusion,  as  the  author  termed  it,  “LCD  is  itself  a work-in-progress”  (Parchoma,  2003.  p52).  Five 
phases  of  ADDIE  was  not  a once-and-for-all,  but  always  ongoing,  cyclic  process:  including  leaner  profile  analysis, 
alignment  of  course  structure,  content,  features,  and  delivery  modes  with  learner’s  needs  and  desires,  task  analysis, 
constant  user  evaluation,  continued  revision  based  on  evaluation  and  feedback. . . 

Analysis  Of  This  Approach. 

It  seems  that  individualized  instruction  means  a lot  of  efforts  for  the  instructor  and  instructional  designers. 
Perhaps,  it  is  not  practical  and  economical  to  achieve  at  all.  Yes,  it  is  true  that  a lot  of  deliberations  and 
considerations  should  be  taken,  but  it  might  be  the  only  strategy  to  win  an  edge  over  other  designs  in  the  online 
learning  market. 

This  decision-upon-input  mechanism  doesn’t  mean  everything  is  fluid.  As  Parchoma  (2003)  concluded  at 
the  end  of  the  article,  “many  appropriate  LCD  choices  are  temporal  and  iterative.  The  basis  for  making  LCD 
choices — matching  detailed  learner  profiling  and  task  analyses  with  effective  pedagogical,  administrative,  and 
technological  choices — remains  relatively  stable. ”(p52).  The  following  section  will  present  a successful  adaptive 
“engine”  that  employs  three  models  and  metadata  to  design  and  deliver  personalized  learning. 

Adaptive  Personalized  E-learning 

Inspired  by  the  Intelligent  Tutoring  Systems  (ITS),  Conlan,  Wade,  Bruen,  & Gargan  (2002)  presented  an 
engine,  namely  Adaptive  Hypermedia  Systems  (AHS).  By  merging  three  models:  content,  learner,  and  narrative 
modeling,  this  engine  is  driven  by  metadata.  The  definitions  of  metadata  and  the  three  models  are  described  in  detail 
as  follows. 

The  basic  idea  of  the  content  model  is  that  the  engine  should  have  a repository  of  reusable  learning  objects 
(LO).  The  learning  objects  must  be  fine-grained  enough  to  be  reusable,  “the  smaller  the  granularity  of  the  content 
the  greater  potential  exists  for  the  LOs  reuse”  (Conlan  et  al.,  2002.  pl02).  In  addition,  each  unit  has  to  contain 
enough  information  so  that  each  of  them  is  meaningful  and  can  be  easily  sequenced  with  other  units  to  make  a 
coherent  and  logical  flow.  Usually,  each  LO  represents  one  learning  objective.  Another  important  concept  in  the 
content  model  is  “candidate  content  group”.  “Each  candidate  content  group  contains  learning  objects  that  fulfill  the 
same  content  requirement”  (Conlan  et  al.,  2002.  pl03).  This  way,  the  repository  consists  of  clusters  of  LOs  of 
similar  content  subject  while  the  subject  of  each  cluster  is  different  from  that  of  any  other  cluster. 

Metadata  in  the  content  model  “describe  both  technical  and  pedagogical  aspects  of  the  LO.  This 
information  is  only  useful  to  a course  designer  in  selecting  appropriate  learning  content,  but  can  be  used  by  an 
adaptive  engine  to  select  appropriate  content  where  there  may  be  many  candidate  LOs  available  to  fulfill  a learning 
or  technical  requirement”  (Conlan  et  al.,  2002.  pi 01). 

The  learner  model  stores  the  predicted  learner  characteristics  and  pedagogical  strategies.  Some  mechanism 
is  used  to  “populate  the  model”  (Conlan  et  al.,  2002.  pi 03),  such  as  a pre-test  or  questionnaire,  inquiring  learners’ 
prior  knowledge,  learning  objectives,  and  learning  style  preferences.  Such  information  is  uploaded  and  stored  in  the 
model  so  that  “the  system  can  consult  with  the  learner  model  to  adapt  the  performance  of  the  system  to  each 
student’s  characteristics”  (Conlan  et  al.,  2002.  pl03). 

The  narrative  model  decides  what  instructions  should  be  delivered  based  on  the  information  the  system 
gathered  from  the  learner,  “the  narrative  model  for  a course  describes  the  rules,  developed  by  domain  and 
pedagogical  experts,  which  govern  the  range  and  scope  of  personalized  course  that  the  adaptive  engine  can  produce 
for  learners”  (Conlan  et  al.,  2002.  pi 05).  Congruency,  coherency  and  contextualization  are  the  three  principles  that 
the  model  follows.  The  course  should  be  structured  in  the  way  that  is  congruent  with  expert’s  knowledge  of  the 
domain.  The  arrangement  of  learning  objects  should  be  coherent  and  can  fulfill  the  learning  goal  of  the  user. 

Another  important  consideration  of  the  model  is  contextualization,  “to  achieve  effective  learning  this  personalization 
should  put  the  content  in  a context  that  the  student  can  understand  and  to  which  they  can  relate”  (Conlan  et  al.,  2002. 
p!03). 
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This  adaptive  hypermedia  services  were  put  into  use  for  the  design  and  delivery  of  an  undergraduate 
computer  course.  Upon  completion  of  the  course,  students  were  surveyed  regarding  to  their  perceptions  and 
satisfaction  level  with  the  course.  87%  of  the  students  were  happy  with  how  the  content  was  structured  in  the 
personalized  courses,  which  indicated  a huge  success  for  the  first  trial  of  this  design. 

Analysis  of  this  approach 

One  feature  the  author  emphasized  again  and  again  in  this  system  is  that  all  decisions  about  the  design  and 
delivery  of  the  course  were  made  at  runtime.  The  adaptive  system  was  designed  in  the  way  that  personalized 
instruction  is  created  automatically  in  real  time.  This  design  approach  bears  a lot  of  similarity  with  Merrill’s  (1999) 
instructional  transaction  theory  (ITT).  The  goal  of  ITT  is  to  realize  adaptive  automatic  instructional  design  in  an 
interactive  learning  environment.  The  theory  focuses  on  the  following  three  aspects: 

S The  idea  of  transaction:  transaction  is  algorithm-controlled  interaction,  in  other  words,  the  sharing 
of  leadership  between  learner  and  instructor. 

S Knowledge  objects:  knowledge  objects  are  interrelated  elements  of  knowledge.  There  are  four  kinds 
of  knowledge  objects:  entity,  property,  activity  and  process. 

S Classes  of  instructional  transactions:  there  are  13  classes  of  instructional  transactions,  including: 
identify,  execute,  interpret,  judge,  classify,  generalize,  decide,  transfer,  propagate,  analogize, 
substitute,  design  and  discover. 

The  instructional  transaction  theory  was  a breakthrough  towards  realizing  individualized  instruction. 
However,  the  “expert”  system  of  transactions  that  makes  all  the  decisions  regarding  to  the  design  and  development 
of  the  instruction  takes  too  many  considerations  into  account.  Thus,  building  such  a system  is  too  formidable  a task 
for  many  instructional  designers.  Nevertheless,  the  success  of  this  adaptive  hypermedia  system  presented  above  may 
herald  the  real  arrival  of  the  era  for  individualized  instructions. 

Several  Principles  of  Individualized  Instruction  Design  and  Development 

Based  on  previous  literature  and  success  stories,  the  following  principles  can  be  concluded  as  germane  to 
the  design  and  development  of  individualized  instructions:  1).  constant  transaction/dialogue  between  the  proctor, 
instructional  designer  and  the  learner  (or  shared  control  between  learners  and  instructional  designer,  proctors);  2). 
breaking  down  domain  content  into  meaningful  units;  structuring  the  units  into  a coherent  flow  and  contextualizing 
the  flow  according  to  learner  characteristics;  3).  mechanisms  of  motivating  learners;  4).  metacognitive  training  to 
help  students  to  develop  self-direction. 

Transaction/dialogue  And  Shared  Control 

Constant  transactions  between  the  proctor,  instructional  designer  and  the  learner  will  equip  the  designer  and 
proctor  with  rich  information  about  a student’s  characteristics,  needs,  wants,  learning  styles,  and  desires.  Dialogues 
between  the  two  parties  can  easily  form  a feedback  loop,  so  that  the  proctor  can  monitor  the  learner’s  progress  and 
motivation  level  and  therefore  quickly  adjust  the  instruction.  Built  based  on  Skinner’s  social  constructivist  theory, 
Keller’s  (1968)  personalized  system  of  instruction  also  involved  a “high  level  of  student-student  and  student- 
instructor  interactions”  (Pear  & Crone-Todd,  2002).  Beyond  these  pedagogical  concerns,  frequent  dialogues  can 
affect  students  emotionally  and  can  help  students  to  overcome  the  sense  of  isolation.  Dialogue  was  proposed  to  be  a 
very  important  variable  for  the  independent  learning  in  Moore’s  (1993)  theory  of  transactional  distance  (Garrison, 
2000).  Learner  involvement  in  distance  instructional  design  conforms  to  concurrent  distance  design  theories. 

Garrison  (2000)  pointed  out  that  there  was  an  identifiable  shift  from  structural  to  transactional  theoretical 
framework  and  models.  Among  the  transactional  theories,  “shared  control  was  seen  to  be  reflective  of  the 
transactional  nature  of  an  educational  experience” (Garrison,  2000)  (p7).  Nevertheless,  teacher-learner  dialogue  is 
not  the  end  of  the  shared-control  story.  Control  embodies  much  more  meaning  than  that.  “The  control  model  places 
within  the  macro  structural  level  of  teacher,  student  and  content  the  micro  level  transactional  elements  of  proficiency 
(ability  and  motivation),  support  (human  and  non-human  resources),  and  independence  (opportunity  to 
choose)”(Garrison,  2000.  p7).  Moreover,  under  such  a shared-control  circumstances,  the  learner  should  not  only 
choose  his/her  learning  objectives,  but  also  can  choose  the  pacing  rhyme  which  is  comfortable  to  him/her.  The 
ability  to  allow  student  to  self-pace  his/her  own  learning  was  regarded  as  one  of  the  defining  features  of  a 
personalized  system  of  instruction  (PSI)  (Grant  & Spencer,  2003). 

Learning  Units  And  Motivation  Concerns 

Besides  self-pacing,  Grant  & Spencer  (2003)  determined  that  “unit  mastery  requirement”  to  be  another 
defining  feature  of  a personalized  system  of  instruction.  “In  a PSI  course,  content  is  separated  into  portions  called 
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units.  To  advance  from  one  unit  to  the  next,  students  must  demonstrate  that  they  have  learned  the  unit’s  material” 
(Grant  & Spencer,  2003.  p2).  The  concept  of  breaking  down  learning  contents  into  units  coincides  with  the 
“knowledge  object”  idea  in  Merrill’s  instructional  transaction  theory  and  the  idea  of  learning  objects.  It’s  not 
surprising  to  find  this  coincidence  because  it  is  much  easier  to  organize  small  chucks  of  information  than  to 
rearrange  a single,  bulky  piece  of  non-delineated  material. 

Different  users  can  come  to  the  instruction  with  a variety  of  differences,  including  prior  knowledge  and 
learning  goal.  One  important  method  of  sustaining  students’  motivation  is  to  present  topics  they  evaluate  highly 
(Rheinberg,  Vollmeyer,  & Rollett,  2000).  Proper  structuring  and  contextualization  are  very  important  too  because 
1).  the  structuring  of  content  units  should  reflect  the  expert’s  knowledge  of  the  domain,  so  that  no  false  presentation 
is  offered  to  the  learner;  2).  it  is  possible  to  buttress  students’  motivation  (one  source  of  positive  incentives  is  “the 
mental  interaction  with  the  learning  material  makes  a coherent  structure  out  of  many  linked  details”  (Rheinberg  et 
al.,  2000.  p518)). 

Metacognitive  Instruction 

The  ability  of  self-regulation  is  seen  as  the  one  of  “our  most  important  quality  as  humans”  (Zimmerman, 
2000)  (pi 3).  The  ultimate  goal  of  higher  education  is  to  develop  students  into  self-regulated  learners.  Helping 
students  to  attain  self-regulation  should  be  the  goal  of  online  learning  as  well.  The  introduction  above  about 
Merrill’s  transaction  and  shared  control  may  have  given  readers  an  impression  that  the  individualized  instruction  is 
designed  rigorously  according  to  the  prescribed  rules  where  instructors  have  complete  control  over  the  structure  of 
the  presentation.  Based  on  their  research  findings,  Lee  & Gibson  (2003)  suggested,  “it  is  desirable  for  students  to 
have  an  opportunity  to  assume  some  direction  related  to  their  learning.  Instructors  should  modify  traditional 
authoritative  roles  and  interact  with  participants  in  a facilitator  role  to  create  such  an  environment”  (pi 85).  As 
mentioned  above,  Dillon  & Greene  (2004)  also  advocated  that  we  should  teach  learner  how  to  learn.  It  is  very 
desirable  that  a balance  should  be  considered  and  achieved  between  structure  and  allowance  for  students’  self- 
direction  and  -regulation  in  an  individualized  instructional  design. 

Summary 

This  paper  has  reviewed  efforts  that  have  been  made  toward  the  design  and  development  of  individualized 
online  instructions  in  higher  education.  Individualized  instruction  design  should  take  into  account  students’ 
differences,  needs,  desires  and  preferences,  endorse  learner-instructor  shared  control,  promote  transactions  among 
different  parties  of  users,  instructors,  and  instructional  designers,  allow  for  easy  modification  of  learning  content, 
and  warrant  quick  adjustment  upon  feedback  from  the  leaner.  In  addition  to  these  requirements,  considerations 
should  be  given  to  motivation  and  metacognitive  concerns. 
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Abstract 

This  purpose  of  this  study  was  to  understand  the  impact  of  scaffolding  through  constraint-based  discussion  forum  on 
high  school  student's  intercultural  collaboration  and  learning.  A total  of  35  intercultural  collaboration  dyads 
participated  in  this  study.  They  engaged  in  a series  of  knowledge  co-constructing  discussions  about  understanding 
different  classroom  cultures,  which  was  mediated  by  a constraint-based  discussion  forum.  Findings  reveal  that  there 
was  a mismatch  between  the  student's  intention  and  their  discussion  message  type,  limited  amount  of  interaction, 
and  existing  discrepancy  between  student's  expectation  and  the  goal  of  the  project.  Implications  and  suggestions  for 
implementation  of  constraint-based  discussion  boards  are  discussed. 

Introduction 

The  advent  of  telecommunication  technologies  has  expanded  our  ability  to  interact  with  others  from  a 
distance  in  a more  efficient  manner.  They  have  also  enhanced  students’  learning  experience  of  engaging  in 
intercultural  learning  activities  by  providing  the  opportunity  for  direct  communication  and  interaction  with  peers 
from  other  countries  or  cultures.  However,  simply  pairing  students  up  and  assigning  them  collaborative  tasks  does 
not  guarantee  the  yield  of  high  quality  conversations  and  favorable  learning  outcomes,  such  as  knowledge  that  is  co- 
constructed from  both  learners  (Lu,  W.,  Diggs,  L.L.,  & Wedman,  J.,  2004).  Moreover,  creating  shared  understanding 
through  consensus  building  is  a complex  cognitive  task.  Effective  collaboration  in  knowledge  co-construction 
occurs  in  the  inquiry  process  of  consensus  building,  which  is  manifested  by  students’  elaboration,  justification,  and 
explanation  of  their  ideas  (Roth  and  Roychoudhury,  1992).  In  order  to  support  the  social  negotiation  of  meaning 
making,  it  is  essential  to  state  the  reasoning  explicitly.  Yet,  not  everyone  is  use  to  or  is  proficient  in  articulating  his 
or  her  thinking.  Thus,  there  is  a need  to  scaffold  people  not  only  in  how  to  negotiate  effectively  in  co-constructing 
meaning  but  to  also  scaffold  the  ability  to  explicitly  express  one’s  own  cognitive  reasoning.  Nevertheless,  unlike 
face-to-face  conversations  where  verbal  and  non-verbal  cues  are  prevalent,  such  as  a quizzical  expression  indicating 
the  need  for  more  elaborative  description,  in  a computer- supported  learning  environment  almost  all  communication 
is  text-based.  For  this  reason,  scaffolding  plays  an  even  more  crucial  role  in  knowledge  co-construction  in  online 
learning  environments. 


Scaffolding  in  Online  Discussions 

In  this  study,  two  types  of  scaffolding  were  employed  to  promote  shared  meaning  making  and  knowledge 
co-construction  among  learners  in  a web-based  asynchronous  conferencing  environment.  The  first  type  of 
scaffolding  is  argumentation  scaffolding.  According  to  Jonassen  and  Remidez  (2002)  “argumentation  is  a 
fundamental  process  of  social  negotiation  through  informal  reasoning  (p.  237).”  However,  they  further  explained, 
that  for  collaborative  groups  to  reason  collectively,  explicating  informal  reasoning  is  especially  necessary  in  the 
process  of  the  argumentation.  In  addition,  they  asserted  that  technology  has  the  capability  to  facilitate  such  social 
negotiation  by  means  of  computer-supported  collaborative  argumentation  (CSCA).  Hence,  Jonassen  and  Remidez 
identified  constraint-based  discussion  forums  as  a type  of  CSCA  scaffold  in  which  they  described  as  pre- structured 
forms  of  conversation  that  are  imposed  onto  the  discussion  with  pre-classified  conversational  attributes  to  fit  the 
structure  constraining  the  nature  of  verbal  interactions  among  participants.  Furthermore,  unlike  threaded  discussion 
forums,  constraint-based  CSCA  has  the  capability  of  embedding  multiple  prompting  questions  into  a conversation 
through  which  learners  are  guided  in  externalizing  their  reasoning.  Thus,  constraint-based  CSCA  can  be  considered 
a method  of  operationalizing  conceptual  and  strategic  scaffoldings  of  informal  reasoning. 
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The  second  type  of  scaffolding  is  collaborative  knowledge-building.  A model  developed  by  Stahl  (2000) 
that  represented  the  cycle  of  personal  and  social  knowledge  building  was  adapted.  However,  in  this  study  we  only 
focused  on  the  social  knowledge  building  process.  Basically,  Stahl  introduced  a number  of  important  phases  in  the 
process  of  collaborative  knowledge-building  in  which  it  initially  begins  with  learners’  personal  beliefs.  Figure  1 is  a 
diagram  that  shows  the  adapted  Stahl  model  in  addition  with  examples  of  argumentation  scaffolding  that  can  be 
implemented  in  a constraint-based  discussion  forum  used  in  an  intercultural  learning  activity.  Hence,  this  study 
aimed  to  understand  the  impact  of  scaffolding  through  constrain-based  discussion  forum  on  high  school  student’s 
intercultural  collaboration  and  learning. 


Figure  1.  Adaptation  of  Stahl’s  model  of  the  social  knowledge-building  process  in  CSCA 

Method 

This  study  was  part  of  a larger  project,  the  Intercultural  Communication  Over  the  Net  (ICON)  project, 
which  was  a transnational  collaborative  project  between  one  high  school  in  the  U.S.  and  two  high  schools  in  Taiwan 
that  was  facilitated  in  the  beginning  of  the  year  2004  by  the  College  of  Education  at  the  University  of  Missouri  - 
Columbia,  USA  and  the  Graduate  Institute  of  Education  in  National  Chiao  Tung  University,  Taiwan.  Similar  to 
other  transnational  or  international  network  projects  via  technology,  such  as  emailing  or  video  conferencing,  the 
ICON  project  provided  the  opportunity  for  participating  teachers  and  students  to  build  local-to-global,  cross-cultural 
understandings  through  the  interaction  and  collaboration  with  people  from  another  country  or  culture. 

However,  the  ICON  project  was  unique  in  the  sense  that  constraint-based  discussion  boards  were  used  as 
the  primary  tool  for  communication  between  the  students.  Moreover,  the  project  researchers  were  not  only 
responsible  for  creating  the  constraint-based  discussion  board  topics  and  questions  but  they  also  designed  and 
developed  the  interdisciplinary  instructional  materials  that  were  used  in  the  project. 

Participants 

The  participants  consisted  of  162  students  (64  American  and  98  Taiwanese)  from  three  different  high 
schools  in  the  US  and  Taiwan.  They  were  randomly  assigned  to  one  of  the  two  conditions;  either  collaborating  with 
a learning  partner  from  the  same  culture  or  collaborating  with  a learning  partner  from  a different  culture.  In  this 
study,  we  only  focused  on  the  interaction  between  the  students  that  were  assigned  to  the  intercultural  collaboration 
dyads,  which  were  35  dyads  in  total.  The  participants  from  Taiwan  were  students  enrolled  in  an  English  class  and 
the  participants  from  Missouri  were  students  enrolled  in  a Theology  and  Social  Justice  course.  Also,  both  the 
Taiwan  and  American  participants  were  provided  with  Tablet  PCs  that  had  wireless  Internet  access  and  they  all 
possessed  sufficient  computer  skills. 
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Materials:  Collaboration  Task 

The  main  objective  of  the  project  was  to  introduce  the  students  in  a systematic  way  to  different  cultural 
traditions  and  norms  as  compared  to  their  own  so  to  foster  intercultural  insight,  understanding,  and  sensitivity. 
Moreover,  it  was  also  decided  that  the  context  would  easily  relate  to  real  life  experiences  so  that  the  students  were 
encouraged  to  examine  and  reflect  on  their  own  perspectives,  attitudes  and  cultural  background.  Therefore,  the 
overall  theme  for  the  instructional  activities  was  “Going  abroad  to  study”  and  the  sub-topic  was  “Classroom 
Manners”. 

At  first,  the  students  were  shown  a video  clip  presenting  a scenario  of  a prospective  exchange  student 
preparing  to  study  abroad  (i.e.,  either  Taiwan  or  America).  In  the  scenario,  the  prospective  exchange  student  had 
asked  advice  from  a classmate  that  was  a former  exchange  student.  The  conversation  between  the  two  students 
ended  with  a statement  made  by  the  former  exchange  student  in  regards  to  cultural  differences  in  classroom  settings. 
After  watching  the  video  clip,  the  students  were  then  asked  to;  1).  interpret  the  meaning  of  the  statement  made  by 
the  former  exchange  student,  2).  suggest  what  questions  should  the  prospective  exchange  student  ask,  and  3).  seek 
answers  to  the  three  most  important  questions.  At  the  end  of  this  activity,  the  students  were  asked  to  work  together 
to  create  a web  page  displaying  the  outcome  of  their  collaborative  discussions. 

Procedure 

Prior  to  working  on  the  instructional  unit,  the  participants  were  instructed  on  the  use  of  the  constraint-based 
discussion  board.  They  also  spent  a few  weeks  practicing  with  a warm  up  discussion  board  activity  (i.e.,  What’s  in  a 
name?)  in  order  to  not  only  get  themselves  familiarized  with  the  networked  learning  system  but  to  also  acquaint 
themselves  with  their  learning  partners  from  the  other  culture.  Three  constraint-based  discussion  boards  regarding 
the  main  topic  “Classroom  Manners”  (i.e.,  Understanding  why,  Asking  questions,  and  Answers  for  the  questions) 
were  then  set  up  with  minor  differences  in  terms  of  message  types  and  structures. 

The  instructional  module  was  carried  out  for  three  weeks.  Every  student  was  required  to  login  to  the  system 
to  check  the  discussion  boards  every  other  day.  At  the  end  of  the  first  two  weeks,  each  dyad  had  to  reach  a 
consensus  regarding  the  content  of  their  collaborative  discussions.  In  the  third  and  last  week,  each  dyad  had  to  then 
collaboratively  create  their  final  product,  which  was  a “Going  abroad  to  study  Q&A”  web  page,  based  on  their  co- 
constructed knowledge  and  understanding. 

Context:  CSCA  Environment 

The  scaffolded  discourse  environment  used  in  the  study  was  modified  from  a threaded  asynchronous 
conferencing  forum.  When  creating  a new  discussion  forum  in  this  environment,  the  instructors  were  able  to  define 
the  message  types,  create  the  input  format,  establish  prompting  questions  for  each  message  type,  and  specify  the 
relationship  between  the  message  types.  Instructors  had  the  flexibility  of  creating  the  structure  based  on  the 
objectives  of  the  instructional  units.  Figure  2 shows  one  of  the  structures  that  were  implemented  in  the  study.  In  this 
example,  7 types  of  message  were  created. 

When  posting  messages  to  the  constraint-based  discussion  board,  students  first  need  to  choose  a message 
type  in  order  to  initiate  the  discussion;  then  they  are  required  to  type  in  their  message  content  according  to  the 
prompting  questions.  The  available  message  types  depended  on  the  type  of  the  message  that  they  were  replying  to. 
For  instance,  figure  3 shows  an  example  of  the  pre- structured  conversation  form.  The  constraint  of  replying  message 
types  and  the  guidance  of  message  content  enforced  students  to  carry  out  informal  reasoning  and  explicitly  express 
their  thinking. 

Data  Collection  and  Analysis 

Data  was  collected  through  three  main  sources,  which  were  observation  from  the  constraint-based 
discussion  boards,  students’  final  web  pages  as  artifacts,  and  the  focus  group  that  was  conducted  after  the 
instructional  module  was  completed.  The  data  analysis  strategies  used  in  this  study  included  content  analysis, 
artifact  assessment,  and  inductive  analysis.  The  body  of  discussion  threads  were  collected  and  analyzed  by  a 
framework  adapted  from  van  Boxtel  (2001/2002)  including  the  amount  and  type  of  student  discourse  about  the 
problems  that  has  to  be  solved,  individual  statements,  positions  related  to  the  individual  statements,  elaboration, 
clarification  request,  and  explanation.  The  students’  web  pages  were  graded  using  a rubric  generated  by  the 
researchers.  The  focus  group  transcription  was  aggregated,  open  coded,  and  consequently  themes  had  emerged. 


Message 

Type 

Description 

Prompt  Questions 

Interpretation 

Please  use  this  message  type  to  post  your 

In  your  own  words,  please  write 
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interpretation  regarding  what  Jenny 
(character  in  the  scenario)  said. 

down  one  of  your  interpretations. 

Disagree 

Please  use  this  message  type  if  you  disagree 
with  the  posting. 

What  do  you  disagree  with? 
Why  do  you  disagree  with  it? 
How  would  you  revise  the 
interpretation? 

Agree 

Please  use  this  message  type  if  you  agree 
with  the  posting. 

What  do  you  agree  with? 
Why  do  you  agree  with  it? 

Clarification 

Request 

Please  use  this  message  type  if  you  do  not 
understand  the  posting  and  need  more 
information  about  it. 

What  part  did  you  not  understand? 

Others 

Please  use  this  message  type  if  you  have 
something  to  say  but  does  not  fall  into  the 
other  types. 

What  other  things  did  you  think  of? 

Response 

Please  use  this  message  type  if  you  want  to 
reply  to  a clarification  request. 

Please  answer  the  questions  that  have 
been  addressed  by  your  learning 
partner. 

Support 

Please  use  this  message  type  if  you  want  to 
add  an  idea  or  comment  to  a posting  that  you 
agree  with. 

Please  give  an  example  or  comment 
that  support  what  has  been  written. 

Figure  2.  An  example  of  message  types,  description  of  the  types,  and  associating  prompt  questions 


Agree 

Disagree 

Clarification 

Request 


Agree 

Disagree 

Clarification 

Request 


Figure  3.  Structure  of  the  relationship  among  message  types 

Results  and  Discussion 

Mismatching  of  student  intension  and  message  type.  From  the  observation  of  the  discussion  threads,  we 
found  that  the  students  did  not  select  the  appropriate  message  type  that  correctly  reflected  their  intention  statements. 
Figure  4 is  an  excerpt  from  the  discussion  board  where  students  were  asked  to  interpret  what  they  thought  was  the 
meaning  behind  the  statement  made  by  the  former  exchange  student  in  the  video  clip.  Student  A provided  his  own 
interpretation  of  the  following  statement  “American  students  talk  more  than  teachers  in  class”,  and  student  B replied 
to  student  A’s  interpretation  based  on  her  own  learning  experience  in  American  classrooms.  However,  it  seems  that 
the  second  message  made  by  student  B,  in  which  she  selected  ‘others’  as  the  message  type,  could  be  classified  as  a 
‘disagree’  message  posting  instead.  Another  phenomenon  we  found  was  that  in  some  messages,  the  students  did  not 
provide  statements  in  answer  to  each  prompting  question.  Therefore,  the  embedded  prompting  questions  did  not 
seem  to  enforce  the  students  to  externalize  their  thinking. 

In  the  second  discussion  board  (Classroom  Manners:  Asking  Questions),  we  found  that  the  students  did  not 
follow  the  task  instructions  accordingly.  The  main  task  for  this  discussion  activity  was  that  students  had  to  think  of 
three  questions  that  would  be  most  helpful  for  an  exchange  student  to  ask  prior  to  going  abroad  to  study.  Afterwards, 
they  must  then  come  to  a conclusion  of  the  three  most  appropriate  questions.  However,  the  statements  in  the 
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discussion  threads  did  not  match  the  message  type.  In  addition,  students  tended  to  answer  their  own  questions 
instead  of  negotiating  with  their  learning  partner  on  which  questions  were  the  most  important.  One  possible 
explanation  was  that  students  were  not  utilizing  the  tools  proficiently.  It  seems  that  students  had  no  mental  model  of 
the  discussion  board  structure.  We  suggest  that  future  research  on  constraint-based  discussion  boards  should  provide 
the  big  picture  of  the  structure  so  that  students  will  be  able  to  know  in  advance  what  message  and  responding  types 
are  available.  This  idea  is  similar  to  procedural  scaffolding,  a form  of  scaffolding  identified  by  Hannafin,  Land,  and 
Oliver  (1999)  for  learning  in  an  open-ended  learning  environment.  Based  on  their  classification  of  scaffolding, 
implementation  of  constraint-based  discussion  boards  need  not  only  the  conceptual  and  strategic  scaffoldings,  but 
also  the  procedural  scaffolding  as  well. 


Student  A (Taiwan) 


Tue,  Apr  20,  2004  11:27:21  AM 


Message  type:  Interpretation 


Prompt:  In  your  own  words,  please  write  down  one  of  your  interpretations. 

Because  it’s  different  leading  business  in  class.  In  America  students  are  the  most  important  roles  in  class.  But 
In  Taiwan  is  contrary.  So  they  talk  more  than  teacher. 


Student  B (American) 


Mon,  Apr  26,  2004  8:01:35  PM 


Message  type:  Others 


Prompt:  What  other  things  did  you  think  of? 

Although  it  is  true  that  students  in  America  may  speak  more  in  class  than  students  in  other  countries,  it  is  not  a 
matter  of  disrespect.  In  America,  we  learn  better  when  we  participate  and  take  part  in  class  discussion. 


Figure  4.  An  example  of  discussion  threads 


Limited  amount  of  co-constructed  knowledge.  From  the  analysis  of  the  final  “Going  aboard  Q&A”  web 
pages,  we  found  that  the  student  pairs  separately  listed  two  sets  of  questions  and  answers.  It  was  discovered  that  the 
students  answered  their  own  questions  without  any  negotiation  with  their  learning  partner.  Therefore,  there  was 
limited  evidence  of  knowledge  co-construction  between  the  students.  This  was  supported  by  the  observed  low 
interaction  in  the  discussion  boards. 

According  to  net  etiquette,  it  is  necessary  to  show  courtesy  by  checking  the  discussion  board  regularly  and 
providing  timely  responses,  especially  when  students  are  involved  in  knowledge  co-construction  activities  that 
require  interaction.  In  figure  4,  the  original  message  and  the  reply  message  had  a six-day  time  lapse.  This  did  not 
occur  just  in  this  example.  Unfortunately,  the  lack  of  timely  response  was  commonly  observed  across  all  the  boards. 
From  the  focus  group  interview,  some  students  expressed  that  at  the  beginning  they  were  very  excited  to  participate 
in  the  warm  up  activities,  but  late  or  lack  of  timely  response  had  then  discouraged  them.  Besides,  the  experimental 
learning  module  was  carried  out  during  the  regular  semester.  Although  we  had  tried  to  integrate  the  learning  module 
into  their  course  curriculum,  participants  from  the  schools  still  had  a limited  amount  of  time  that  could  be  devoted  to 
this  learning  experience. 

In  addition,  the  teachers  had  emphasized  too  much  on  completing  the  final  product  rather  than  focusing  on 
the  importance  of  the  learning  process.  Originally,  the  objective  of  the  activity  was  for  the  students  to  bring  up  and 
share  their  own  three  questions  (totaling  six  questions  for  each  dyad)  and  then  decide  together  on  which  three  of  the 
six  were  the  most  important  questions  to  address.  The  value  of  the  task  was  the  process  of  negotiation  between  the 
students  in  trying  to  come  to  a consensus  on  what  was  important  to  know.  However,  due  to  the  lack  of  time  and 
teachers’  misguidance  in  pushing  students  to  quickly  complete  the  task,  the  students  did  not  co-construct  their 
knowledge  but  just  combined  what  they  each  knew  into  one  artifact. 

Discrepancy  between  students’  expectation  and  the  goal  of  the  project.  In  the  focus  group,  some  students 
expressed  that  their  expectation  of  this  project  was  not  fulfilled.  They  felt  that  the  constraint  imposed  on  the 
discussion  board  had  actually  inhibited  their  learning.  Their  first  impression  and  expectations  of  this  project  was  to 
make  friends  from  other  cultures  and  to  satisfy  their  curiosity  about  the  current  life  style  in  Taiwan/America.  Once 
students  found  out  that  there  was  a specific  task  that  required  them  to  work  on  more  than  simply  sharing  ideas,  some 
of  them  did  not  seem  to  be  truly  engaged  in  the  learning  activity.  It  is  suggested  that  empowering  and  encouraging 
students  to  have  some  chitchats  might  stimulate  them  to  develop  intimacy  with  their  learning  partner,  which  might 
promote  more  interaction  and  negotiation  on  a given  task. 


Conclusion 

This  study  examined  the  impact  of  scaffolding  through  constraint-based  discussion  forum  on  high  school 
student’s  intercultural  collaboration  and  learning.  The  findings  suggest  that  other  than  conceptual  and  strategic 
scaffoldings,  students  also  need  procedural  scaffolding  (Hannafin,  Land,  and  Oliver,  1999),  which  can  help  them 
utilize  a constraint-based  discussion  forum  more  adequately.  Besides  taking  the  nature  of  the  task  into  account, 
offering  pre-experiment  training,  providing  sufficient  amount  of  time,  and  emphasizing  net  etiquette,  should  lead  to 
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increased  amount  of  discussion  board  interaction.  Moreover,  in  order  to  highly  engage  students  in  performing  a task, 
understanding  the  students’  needs  and  expectations  are  also  important  issues  to  consider. 
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Abstract 

The  use  of  the  internet  to  deliver  education  has  increased  dramatically  over  the  last  decade.  There  are 
many  statistical  results  that  prove  this  information.  Educational  institutions  offered  distance  education  courses 
additional  for  each  year.  In  addition,  institutions  offered  degree  and  certificate  programs  over  the  Internet. 
Therefore,  two  online  certificate  programs  will  be  analyzed  and  then  instructional  design  model  for  online 
certificate  programs  will  be  proposed  in  this  study. 


Introduction 

There  has  been  a tremendous  problem  around  the  world  in  recent  years.  This  problem  is  related  to  rapidly 
changing  profile  of  human  resource  in  many  fields.  For  example,  a half  life  of  an  engineer's  technical  skills  was 
reported  in  Smerdon's  article  (1996).  Half  life  refers  to  the  length  of  time  it  would  take  for  half  of  everything  an 
engineer  knew  about  his/her  field  to  become  obsolete.  It  takes  7.5  years  for  mechanical  engineers,  5 year  for 
electrical  engineers,  and  2.5  years  for  software  engineers.  It  is  clear  that  job  and  carrier  changes  come  more 
frequently  for  today's  adults  than  ever  before. 

Human  capital  needs  include  requirements  for  employees  to  develop  their  skills,  to  adapt  rapid  change  in 
the  work  place,  and  to  bring  innovation  into  solving  problem.  Organizations,  private  sectors,  and  universities  are 
trying  to  supply  the  demand  for  qualified  employees  in  many  countries.  Thus  technical  training,  graduate  programs, 
and  certificate  programs  are  provided  in  many  ways  for  students  and  employees. 

One  of  the  effective  ways  to  educate  people  especially  adult  is  through  online  learning.  Because,  it  is 
suitable  for  especially  working  adults  who  work  fulltime  and  seek  for  continuous  education  as  a part-time  student 
and  allow  them  to  achieve  their  goals. 

It  can  be  concluded  that  offering  online  certificate  programs  is  one  of  the  reasonable  way  of  updating 
employees’  knowledge  and  skills,  and  providing  improvement  opportunity  for  life-long  learners. 

Purpose  and  Intend 

Two  online  certificate  programs  (Online  Information  Technologies  Certificate  Program,  E-Course 
Certificate  Program)  are  investigated  in  this  study.  Both  have  similar  design  properties  but  we  are  not  sure  that  there 
is  an  exact  model  that  is  followed  while  they  were  developed.  Although  there  are  some  models  used  for  online 
learning,  we  could  not  find  any  specific  model  devoted  to  developing  online  certificate  programs.  In  addition, 
Gustafson  and  Branch  (1997,  p.  78)  states  that  “most  authors  completely  ignore  mentioning  the  conditions  under 
which  their  models  should  be  used”.  Therefore,  we  have  tried  to  develop  a new  ISD  model  for  online  certificate 
programs  based  on  ADDIE  (Beckschi  & Doty,  2000)  and  Rapid  Prototyping  model  (Tripp  & Bichelmeyer,  1990). 

OCEP-ID  Model 

OCEP-ID  Model  explains  process  that  provides  a means  for  decision  making  to  determine  who,  what, 
when,  where,  why,  and  how  questions  of  online  certificate  programs.  The  followings  are  the  steps  of  our  model 
(Figure  1); 

• General  Analysis  (problem,  need,  audience,  task,  job,  content) 

• General  Design  (objectives,  instructional  analysis  and  strategies,  evaluation) 

• Prototype  Loop 

o Design 
o Development 
o Implementation 
o Evaluation 

• General  Evaluation  (Summative) 
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As  mentioned  earlier,  this  model  was  created  with  help  of  ADDIE  and  rapid  prototyping  of  Instructional 
Design  Models.  ADDIE  is  broader  model  and  it  is  not  developed  for  online  learning.  However,  we  thought  that  it 
defines  the  necessary  steps  to  develop  an  ISD  model  as  it  is  systematic  and  well-defined.  We  have  changed 
development  and  implementation  phases  of  ADDIE  model  with  “Prototype  Loop”,  which  was  designed  by  using  of 
Rapid  Prototyping  model. 

Before  offering  a certificate  program  for  a certain  profession,  you  should  ensure  that  there  is  a real  problem 
due  to  the  lack  of  certain  qualified  staff  in  that  specific  profession.  The  existence  of  such  a problem  should  be 
extracted  from  society  through  several  methods,  such  as  communicational  channel,  field  trips,  research  outputs, 
trends,  reports,  etc.  The  exact  problem  should  be  stated  explicitly  based  on  interpretation  of  data  indicating  a real 
problem  in  local  or  global  area.  After  stating  the  problem  explicitly,  next  issue  is  whether  we  can  solve  the  problem 
by  means  of  the  certificate  program. 

Once  you  determine  to  offer  the  certificate  program,  potential  participants  of  that  program  should  be 
decided.  Their  existing  skills,  demographic  data,  knowledge  and  attitude  in  cognitive,  affective,  and  psychomotor 
domain  should  be  analyzed.  Therefore,  you  can  specify  the  profile  of  potential  participants  (e.g.  education  level, 
language,  etc.).  It  is  crucial  to  find  the  ways  for  accessing  them  to  introduce  the  certificate  program. 

In  a certificate  program,  you  know  that  the  participants  gain  specific  knowledge  and/or  skills  about  a 
profession.  In  this  part,  what  will  be  included  in  the  program  in  terms  of  content  (what  kind  of  knowledge  and  skill) 
should  be  clarified.  Clarifying  the  content  involves  in  conducting  job  and  task  analysis.  The  job  analysis  refers  to 
observe  the  expert  to  break  down  job  into  tasks  and  prioritize  tasks.  Each  task  should  be  broke  down  into  steps  by 
prioritizing  them  through  task  analyses.  With  the  reference  of  job  and  task  analyses,  content  is  derived,  prioritized 
and  sequenced. 

With  regard  to  preliminary  analysis  the  main  purpose  of  the  certificate  program  should  be  clearly  stated. 
The  following  phases  of  the  design  will  be  based  on  this  purpose. 
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General  Design 

Analysis  and  design  phases  overlap  at  some  points.  After  defining  problem,  conducting  need  and  audience 
analysis,  you  should  start  the  design  process  as  well.  That  is,  they  are  parallel  processes.  Overall  knowledge  and 
skills  to  be  covered  in  the  program  should  be  defined  and  be  arranged  in  a proper  sequence.  The  specified  content 
should  be  divided  into  a number  of  courses  according  to  the  nature  of  the  content  and  the  target  audience.  The 
content  load  of  the  courses  should  be  fairly  distributed.  General  goals  of  each  course  are  written  explicitly,  that  is, 
what  learners  should  be  able  to  do  when  they  finish  each  course.  At  this  point  the  issues  related  to  the  program  such 
as  the  length  of  the  courses,  the  number  of  courses  in  each  semester,  the  length  of  the  semesters,  the  number  of  the 
semesters,  and  the  length  of  the  program  should  be  decided. 

General  instructional  strategy  to  be  adopted  and  major  evaluation  strategies  should  be  clarified. 

Instructional  strategies  should  be  developed  based  on  course  content  and  objectives  as  well  as  learner  characteristics. 

Prototype  Loop 

Prototype  Loop  covers  design,  development,  implementation  and  evaluation  of  each  semester.  It  means  that 
the  content  of  the  semesters  are  developed  successively  in  this  phase.  To  state  more  clearly,  the  courses  in  each 
semester  are  designed  through  one  cycle  of  the  Prototype  Loop.  Each  of  these  steps  goes  on  partially  parallel  with 
the  prior  and  following  step.  Since  each  semester  may  consist  of  more  than  one  course,  the  courses  are  designed  in 
parallel  in  the  prototype  loop.  In  other  words,  going  through  the  Prototype  Loop  once  means  that  you  design, 
develop,  implement,  and  evaluate  a semester. 

The  components  of  Prototype  Loop: 

Design:  In  this  step  the  following  tasks  should  be  done  for  each  course  for  preparing  a semester  in  the 
program; 

• Content  outline 

• Specific  objectives  (what  specific  outcomes  should  be  reached) 

• Describe  the  set  of  materials  you'll  use 

• Determine  instructional  strategies 

• Decide  evaluation  strategies  based  on  objectives,  course  content  and  instructional 

strategy. 


Development:  After  describing  objectives,  materials,  instructional  strategies,  and  evaluation  strategies  for  a 
semester,  it  is  now  time  to  develop/select  them.  There  are  advantages  and  limitations  of  using  online  learning 
environment.  Therefore  the  materials  should  be  selected  from  the  market  or  developed  based  on  these 
considerations.  Some  of  examples  are  listed  below: 

• For  presenting  content;  examples,  simulations,  animations,  video,  tutorial,  interactive 
examples,  summary,  games,  demonstrations, 

• For  supplying  supported  material;  links,  references,  textbooks,  handout, 

• For  preparing  interface;  images,  site-map,  help, 

• For  feedback;  quizzes,  homework,  study  guide 

• For  learner  and  program  evaluation;  tests,  survey  items,  projects. 

Using  an  infrastructure  such  as  Learning  Management  System  could  be  helpful  in  hosting  all  the  components  of  the 
program. 

Implementation:  which  techniques  of  instruction,  what  modes  of  instruction,  which  media,  and  which 
follow-up  strategies  will  you  employ  to  deliver  instruction  are  the  main  questions  in  this  step.  No  matter  how  well 
you  develop  instructional  materials,  the  way  you  use  them  in  instructional  process  determines  the  effectiveness  of 
the  instruction.  In  other  words,  unless  you  apply  appropriate  instructional  strategies  for  your  audience,  you  will  most 
probably  fail  in  achieving  overall  program  goal.  The  audience  may  be  heterogeneous  in  terms  of  their  characteristics 
and  almost  all  of  them  are  adult  learners.  Therefore,  andragogy  is  a major  theory  that  helps  us  to  construct  all 
implementation  steps  including  instructional  strategy.  Andragogy  states  that  "adult  learners  take  control  of  their 
education  and  want  to  opportunity  to  learn  at  their  own  page,  at  times  and  places  compatible  with  the  commitments 
of  family,  work,  and  leisure.  Also,  they  demand  relevant  and  applicable  coursework  and  a learning  environment  that 
is  supportive  and  collaborative."  (Knowles,  1984,  p.  125) 

So,  the  instructional  strategies  are  selected  among  a variety  of  techniques  and  modes  including  problem 
based  learning,  collaborative  learning,  authentic  learning,  role  playing,  group  activities,  individualized  instruction, 
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discussion,  etc.  Note  that  most  of  the  mentioned  methods  are  mainly  constructivist  approach  that  is  favorable  for 
adult  learning. 

The  customization  is  an  important  necessity  for  online  learners.  The  industry  requires  huge  number  of 
employees  who  can  take  initiative,  think  critically,  cope  with  teamwork,  and  solve  problems.  To  meet  this  need  of 
industry  and  life-long  learners,  educational  programs  have  to  focus  on  learning  rather  than  standardization  or  sorting 
people.  In  other  words  the  instructional  design  model  should  focus  on  helping  all  learners  reach  their  potential 
through  customization  (Reigeluth,  1999). 

The  learner  support  and  feedback  system  should  be  handled  carefully  to  overcome  the  disadvantages  of 
online  learning  environment  such  as  lack  of  face  to  face  communication.  On  time,  correct,  suitable  feedback  has 
great  importance  in  order  not  to  de-motivate  learners.  The  learners  should  not  be  allowed  to  alienate.  Additional 
full/part-time  staff  could  be  hired  for  this  purpose  as  instructors  may  not  deal  with  all  the  feedback  requests. 

The  media  selection  is  another  important  issue  in  implementation  step.  As  the  technology  has  advanced,  the 
text  based  materials  on  the  web  are  abandoned  rather  multimedia  components  such  as,  audio,  video,  simulation,  etc. 
and  the  collaborative  tools  such  as  chat,  discussion  group,  e-mail,  audio/video  conferencing  are  embedded  into  the 
instructional  materials  delivered  online.  Do  not  compel  you  to  use  all  media  where  you  need  face  to  face 
communication.  You  may  also  arrange  face  to  face  sessions  at  the  beginning  and  at  the  end  of  semesters.  In  fact, 

“the  trend  in  technology  application  is  toward  hybrid  courses,  courses  that  combine  some  face-to-face  time  with 
online  experiences;  this  is  much  the  same  as  the  trend  in  corporate  training  toward  “blended  learning.”  (Molenda  & 
Sullivan,  2003,  p.6). 

Evaluation:  this  step  is  the  last  one  in  the  Prototype  Loop.  This  evaluation  has  two  functions.  First,  it  has 
summative  evaluation  function  for  the  semester  which  the  loop  is  used  for  developing.  You  decide  the  working  or 
not  working  aspects  of  the  term.  This  will  affect  the  same  semester  of  the  following  application  of  the  certificate 
program  such  as  updating  content  of  the  same  course.  Second,  it  has  formative  evaluation  function  which  affects  the 
following  semester  design  such  as  increasing  online  chat  hours  in  the  courses  to  supply  students’  demands  of  more 
and  longer  chat  sessions. 


General  Evaluation 

The  evaluation  is  related  to  the  overall  system  effectiveness.  The  data  mainly  come  from  Prototype  Loop 
components,  those  are  design,  development,  implement,  and  evaluate.  For  example,  in  each  semester  there  might  be 
complaint  for  page  layout  of  the  courses. 

There  is  a need  to  learn  to  what  extend  the  learners  attain  general  objectives.  Since  this  is  a certificate 
program  a project  that  covers  all  the  program  content  could  be  assigned  to  assess  overall  effectiveness  of  the 
program. 

Conclusion 

This  model  was  prepared  based  on  a real  example  (Online  Information  Technologies  Certificate  Program). 
The  model  is  developed  specifically  for  online  certificate  programs.  Two  paradigms  which  are  objectivism  and 
constructivism  were  used  in  this  model.  Instructional  System  Development  model  and  Rapid  Prototyping  model 
(Tripp  & Bichelmeyer,  1990)  helped  us  during  proposing  this  model.  We  tried  to  use  advantages  of  these  two 
models.  Institutions,  collages,  universities  who  want  to  develop  this  kind  of  programs  may  use  this  model  in  their 
development  phases.  This  model  is  first  draft  and  can  be  tested  and  developed  in  other  cases. 
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Abstract 

A recent  trend  in  the  human  competency  movement  suggests  that  organizations  should  use  the 
characteristics  of  exemplary  performers  as  their  standard  for  hiring  and  training  their  employees.  The  purpose  of 
this  research  paper  is  to  discuss  the  previous  studies  on  differences  between  exemplary  and  average  (fully- 
successful)  performers  and  novice  and  expert  performers.  The  findings  provide  useful  implications  for  trainers  and 
instructional  designers  in  terms  of  what  really  distinguishes  exemplary  from  average  performers  and  the  need  for 
training  people  to  reach  the  level  of  productivity  closer  to  that  of  the  exemplars  instead  of  training  people  to  reach 
merely  minimum  performance  standards. 


Introduction 

White  (1959)  first  introduced  a human  trait  named  competence.  Since  then,  human  competence  has  become 
an  increasingly  important  topic  in  training  and  development.  Much  research  has  been  conducted  with  an  attempt  to 
conceptualize  and  measure  human  competence.  Gilbert  (1978)  suggested  that  there  is  a big  difference  between 
exemplars  and  average  performers  in  their  performance.  He  further  provided  a behavior  engineering  model  that 
provided  an  explanation  of  the  reasons  for  the  performance  differences.  Other  research  studies  provided  different 
perspectives  in  terms  of  what  really  distinguishes  exemplary  from  average  performance.  A recent  trend  in  the  human 
competency  movement  suggests  that  organizations  should  use  the  characteristics  of  exemplars  as  their  standard  for 
hiring  and  training  their  employees.  Therefore,  it  is  important  to  find  out  what  makes  the  exemplars  different  from 
average  performers.  Several  research  studies  on  exemplary  performers  and  their  productivity  differences  compared 
to  average  performers  are  discussed  below. 

Research  Studies  on  Human  Competence 

Gilbert’s  Human  Competence 

Gilbert  (1978)  provided  a creative  theoretic  framework  to  describe,  define  and  measure  human  competence. 
In  the  first  leisurely  theorem,  he  defined  human  competence  as  “a  function  of  worthy  performance  (W),  which  is  a 
function  of  the  ratio  of  valuable  accomplishments  (A)  to  costly  behavior  (B)”  (p.  18).  He  further  explained  that 
competence  is  a “social  concept”  since  it  is  a judgment  about  the  worthy  performance  (p.  29). 

In  the  second  theorem,  Gilbert  (1978)  suggested  that  there  is  a big  difference  between  individual 
performers  in  their  performance.  He  distinguished  people  as  exemplars  versus  typical  performers  and  their 
performance  as  exemplary  performance  versus  typical  performance.  He  further  discussed  the  notion  of  potential  for 
improving  performance  (PIP)  as  the  measure  of  competence,  which  is  the  ratio  of  exemplary  performance  to  typical 
performance.  PIP  serves  two  functions:  it  shows  how  much  competence  we  already  have  and  how  much  can  be 
improved.  PIP  and  competitiveness  are  inversely  related:  the  higher  the  PIP,  the  less  competitive  that  person  or 
group  is;  the  lower  the  PIP,  the  more  competitive  that  person  or  group  is.  Because  there  will  always  be  new 
exemplary  standards,  PIP  is  a “dynamic”  measure. 

Gilbert  (1978)  defined  exemplary  performance  as  “the  worth  of  the  historically  best  instance  of  the 
performance”  (p.  30).  He  later  clarified  this  definition  by  defining  exemplary  performance  as  “the  most  sustained 
worthy  performance  that  we  can  reasonably  expect  to  attain”  (p.  40).  He  explained  that  it  need  not  be  the 
performance  of  historically  best  performer  because  the  exemplar’ s performance  may  be  improved.  He  also  clarified 
that  the  exemplar  must  be  defined  relative  to  a particular  accomplishment  because  the  exemplar  for  one 
accomplishment  may  not  be  the  exemplar  for  another.  Performance  can  be  measures  by  three  major  requirements 
which  are  quality,  quantity  and  cost  (Gilbert,  1978). 

In  the  third  theorem,  Gilbert  (1978)  examined  the  behavior  cause  of  the  differences  in  worthy  performance 
between  exemplars  and  typical  performers.  The  behavior  here  refers  to  the  instrumental  human  behavior  that 
includes  two  aspects:  repertory  of  behavior  (P)  and  a supporting  environment  (E).  He  maintained  that  “for  any 
given  accomplishment,  a deficiency  in  performance  always  has  as  its  immediate  cause  a deficiency  in  a behavior 
repertory  (P),  or  in  the  environment  that  supports  the  repertory  (E),  or  in  both.  But  its  ultimate  cause  will  be  found  in 
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a deficiency  of  the  management  system  (M)”  (p.  76).  Therefore,  in  order  to  improve  a performer’s  competence,  we 
need  to  decide  if  we  should  modify  the  person’s  repertory  of  behavior,  or  change  the  supporting  environment,  or  do 
both.  Gilbert  further  identified  the  components  of  the  behavior  repertory  and  the  environment.  The  three  aspects  of 
behavior  repertory  include  knowledge,  capacity  and  motives,  and  the  three  aspects  of  environment  include  data, 
instruments  and  incentives.  “There  are  great  differences  in  what  people  accomplish,  but  small  differences  in  their 
repertories  of  behavior”  (Gilbert,  1978,  p.  39).  He  suggested  the  analysis  of  the  components  of  the  behavior 
engineering  model  can  be  used  to  diagnose  why  exemplary  performance  is  more  than  typical  performance,  and 
which  behavioral  component  in  the  model  causes  the  PIP.  He  claimed  the  deficiency  in  performance  between  the 
exemplars  and  typical  performers  either  comes  from  its  environmental  support,  or  a person’s  behavioral  repertory,  or 
both. 

During  the  process  of  diagnosing  performance  deficiencies,  Gilbert  (1978)  suggested  motives  and  capacity 
are  usually  the  last  two  reasons  which  cause  incompetence  because  they  seldom  are  the  major  problems.  He  further 
explained  that  the  behavior  engineering  model  itself  gives  an  order  of  the  trouble-shooting  process.  We  should  first 
look  at  the  environmental  variables  beginning  with  data.  We  ask  the  simple  question:  are  there  sufficient, 
informative,  and  reliable  data  to  provide  performers  a guide  to  how  they  should  perform  and  how  well  they  have 
performed  against  a certain  criteria?  He  pointed  out  improper  guidance  and  feedback  are  the  largest  causes  of 
incompetence.  When  we  look  at  instruments,  we  ask  the  question:  do  the  performers  have  the  appropriate  tools  and 
materials  to  work  with?  Next,  we  look  at  incentives  and  ask  the  question  is:  are  there  incentives  contingent  upon 
good  performance?  Finally,  we  look  at  training  as  a possible  solution  to  solve  knowledge  problem.  He  explained  that 
the  behavior  engineering  model  provides  different  ways  to  look  at  behavior  and  to  ask  obvious  questions  in  order  to 
improve  human  competence.  It  helps  to  answer  the  question:  what  might  we  do  to  engineer  superior  performance  or 
exemplary  performance. 

However,  Gilbert  (1978)  pointed  out  that  even  if  we  have  full  knowledge  of  why  the  exemplars  are  superior 
compared  with  typical  performers,  and  even  if  we  give  all  performers  the  information,  instruments,  incentives, 
knowledge,  and  so  on,  of  the  exemplars,  there  will  still  be  some  variance  in  performance.  Someone  will  still  be  an 
exemplary  performer  because  of  some  inherent  characteristics  such  as  quickness,  strength,  intelligence  and 
ambition.  Someone  will  always  find  a better  way  of  doing  it,  have  some  natural  superiority,  or  have  an  unusual 
degree  of  motive  to  excel.  Therefore,  it  is  impossible  to  reduce  PIPs  to  1.0.  That  means,  in  some  situations,  you  may 
train  typical  performers  to  reach  the  level  of  productivity  closer  to  that  of  exemplary  performers,  but  you  may  not  be 
able  to  reach  the  same  level  of  productivity  of  the  exemplars. 

When  comparing  human  competence,  Gilbert  (1978)  maintained  that  we  should  use  exemplary 
performance  as  the  standard  of  comparison.  He  pointed  out  that  oftentimes,  the  average  performance  of  a group  of 
people  is  established  as  the  standard.  The  result  inevitably  leads  to  mediocrity  because  it  reduces  the  potential  for 
improvement.  Using  the  exemplary  performance  as  a comparison  standard  suggests  that  all  performers  would 
achieve  what  the  best  have  achieved  without  considering  the  difference  of  some  inherent  characteristics. 

McClelland’s  Job  Competence  Assessment  Method  (JCAM) 

McClelland  (1973)  challenged  the  value  of  intelligence  testing  and  suggested  that  although  intelligence  is 
important  to  performance,  it  is  other  personal  characteristics,  such  as  motivation,  that  differentiate  successful  from 
unsuccessful  performance  in  a job  role.  McClelland  and  his  associates  used  a research  process  called  Job 
Competence  Assessment  Method  (JCAM)  which  is  a rigorous  empirical  method  for  research  the  job  requirements 
for  exemplary  performance  and  the  attributes  of  the  exemplary  performers.  They  identified  a list  of  characteristics  of 
exemplary  performers  and  the  critical  competencies  that  clearly  differentiate  the  exemplary  performers  from  the 
average  performers.  The  competencies  held  only  by  the  exemplary  performers  are  those  that  distinguish  exemplary 
from  average  performance  (Dubois,  1993). 

Klemp  (1982)  wrote,  “Job  competence  assessment  is  a powerful  new  solution  to  a problem  of  how  to  hire  and  train 
people  for  maximum  effectiveness  ...  the  best  way  to  find  out  what  it  takes  to  do  a job  is  to  analyze  the  job’s 
outstanding  performers  and  then  to  study  what  they  do  that  makes  them  so  effectives”  (p.  55). 

Spenser  and  Spenser’s  Generic  Competency  Models 

Spenser  and  Spenser  (1993)  summarized  20  years  of  research  using  the  McClelland’s  Job  Competence 
Assessment  Methodology  (JCAM)  and  provided  generic  competency  models  for  entrepreneurs, 
technical  professionals,  salespeople,  service  workers  and  corporate  managers. 

Spenser  and  Spenser  (1993)  defined  competency  as  “underlying  characteristics  of  an  individual  that  is 
causally  related  to  criterion-referenced  effective  and/or  superior  performance  in  a job  or  situation”  (p.  9).  The 
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underlying  characteristics  refer  to  five  types  of  competency  characteristics:  motives,  traits,  self-concept,  knowledge, 
and  skill.  According  to  Spenser  and  Spenser  (1993,  p.9-11)  motives  are  “the  things  a person  consistently  thinks 
about  or  wants  that  cause  action”;  traits  refer  to  “physical  characteristics  and  consistent  responses  to  situations  or 
information”;  self-concept  is  “a  person’s  attitudes,  values,  or  self-image”;  knowledge  is  “information  a person  has  in 
specific  content  areas”;  and  skill  is  “the  ability  to  perform  a certain  physical  or  mental  task.”  Among  them, 
knowledge  and  skill  are  more  visible  and  easier  to  develop,  while  motives,  self-concept  and  traits  are  more  hidden 
and  more  difficult  to  develop. 

Spenser  and  Spenser  (1993)  called  exemplars  superior  performers  and  defined  superior  performance  as 
“one  standard  deviation  above  average  performance,  roughly  the  level  achieved  by  the  top  1 person  out  of  10  in  a 
given  working  situation”  (p.  13).  They  pointed  out  that  it  is  differentiating  competencies  that  distinguish  superior 
from  average  performers. 

Spenser  and  Spenser  (1993)  provided  a generic  competency  dictionary,  which  defines  the  scoring  criteria 
for  the  21  competencies  found  most  often,  to  differentiate  superior  from  average  performers  in  286  studies  of 
middle-  to  upper-level  jobs.  The  21  distinguishing  competencies  were  categorized  into  six  clusters.  Achievement 
and  action  cluster  focuses  more  on  task  accomplishment.  It  includes  four  competencies:  achievement  orientation; 
concern  for  order,  quality  and  accuracy;  initiative;  and  information  seeking.  Helping  and  human  service  cluster 
involves  attending  to  other’s  needs,  interest  and  concerns  and  working  to  meet  those  needs.  It  includes  two 
competencies:  interpersonal  understanding  and  customer  service  orientation.  The  impact  and  influence  cluster 
reflects  individuals  concern  with  his  or  her  influence  on  others.  It  includes  three  competencies:  impact  an  influence, 
organizational  awareness  and  relationship  building.  Managerial  cluster  is  a specialized  subset  of  the  impact  and 
influence  competencies,  stating  the  specific  intentions  which  are  especially  important  to  managers.  It  includes  four 
competencies:  developing  others,  assertiveness  and  use  of  positional  power,  teamwork  and  cooperation  and  team 
leadership.  Cognitive  cluster  is  related  to  the  individual’s  effort  to  understand  a situation,  task,  problem, 
opportunity,  or  body  of  knowledge.  It  includes  three  competencies:  analytical  thinking,  conceptual  thinking  and 
technical/professional/managerial  expertise.  Personal  effectiveness  cluster  reflects  certain  aspects  of  an  individual’s 
maturity  compared  to  others  in  relation  to  work.  It  includes  four  competencies:  self-control,  self-confidence, 
flexibility,  organizational  commitment,  and  other  personal  characteristics  and  competencies.  Each  of  the  21 
competencies  includes  a narrative  description  and  a list  of  full-scaled  behavioral  indicators  ranging  from  average 
performance  to  superior  performance.  The  range  of  competency  levels  reveals  the  differences  between  the  average 
and  superior  performers. 

In  addition,  Spenser  and  Spenser  (1993)  also  presented  a series  of  generic  competency  models  which 
describe  the  specific  competencies  of  the  superior  performers  in  technical/professional,  sales,  helping  and  service, 
managerial,  and  entrepreneur  jobs.  The  importance  of  the  competencies  is  different  for  each  job  position.  For 
technical/professionals,  achievement  orientation  is  the  most  frequent  distinguishing  characteristic  of  superior 
technical  contributors.  For  sales  people  and  service  workers,  the  most  important  competency  is  impact  and 
influence.  For  managers,  the  competencies  of  impact  and  influence  and  achievement  orientation  are  of  the  most 
importance.  For  entrepreneurs,  the  most  important  competencies  include  initiative,  seeing  and  acting  on 
opportunities,  persistence,  concern  for  high  quality  of  work,  self-confidence,  monitoring  and  recognizing  the 
importance  of  business  relationship.  Their  findings  showed  that  superior  performance  in  different  job  positions  are 
defined  by  a specific  set  of  competencies  and  the  importance  of  each  competency  is  different  for  different  job 
positions. 

Other  Competency  Studies  for  Various  Job  Positions 

There  are  several  other  studies  focusing  on  specific  job  positions.  Smith  (1992)  presented  a research  study 
which  created  a model  to  describe  the  ideal  performance  of  executives  in  generic  terms.  The  model  described  four 
executive  competencies  which  were  considered  the  most  critical  for  exemplary  executives.  They  include  broad 
industry  knowledge,  self  knowledge,  “big-picture”  thinking  skills,  and  idea-generation  skills. 

Kangis  and  Fago  (1997)  pointed  out  two  basic  qualities  or  competencies  for  exemplary  sales  people  suggested  by 
Mayer  and  Greenberg  (1964): empathy  (“ability  to  feel”)  and  ego  drive  (“ability  to  conquer”).  Empathy  refers  to  the 
individual’s  ability  to  think  the  way  others  are  thinking,  feel  the  way  others  are  feeling,  and  react  accordingly.  Ego 
drive  refers  to  the  salesperson’s  quality  that  makes  him/her  want  to  make  the  sale  for  personal  satisfaction  and 
gratification.  These  two  basic  qualities  make  a top  performer  in  sales. 

Klenke  (2002)  described  the  specific  competencies  possessed  by  three  women  leaders:  Ruth  Simmons, 
Mary  Kay  and  Oprah  Winfrey.  These  competencies  include  transforming/transformational  leadership,  emotional 
intelligence  and  the  ability  to  build  trust. 
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Kasch,  Greenberg,  and  Muenzen  (2003)  reported  that  six  competency  areas  in  hand  therapy  practice  were 
identified  through  a survey  study:  scientific  knowledge,  clinical  judgment/clinical  reasoning,  technical  skills, 
interpersonal  and  communication  skills,  professionalism,  and  resource  management.  Among  them,  clinical 
judgment/clinical  reasoning  and  scientific  knowledge  were  of  the  most  importance.  Professional  practice  exemplars 
and  continuum  of  competence  from  novice  to  expert  were  also  identified. 

Kelley’s  Nine  Strategies  of  the  Star  Performer 

Kelley  (1998;  1999a;  1999b)  used  the  term  “star  performer”  to  refer  to  exemplary  performers.  He  claimed 
that  there  are  no  fundamental  (appreciable  cognitive,  personal-psychological,  social,  or  environmental)  differences 
between  stars  and  average  performers.  What  makes  the  difference  is  how  the  stars  use  what  they  have  in  their  heads, 
not  what  they  have.  Therefore,  star  performers  are  not  born,  instead,  they  are  made.  After  10  years  of  research  and 
practice,  he  and  his  colleague  discovered  nine  work  strategies  which  are  common  to  star  performers,  and  these 
strategies  proved  to  be  effective  in  improving  productivity.  These  strategies  help  to  answer  the  question  of  what  it 
takes  to  become  a star  performer.  He  asserted  that  star  performers  work  very  differently  from  average  performers. 
Kelley  (1999a,  p.  31-35)  identified  nine  strategies  that  star  performers  commonly  use: 

1.  “Initiative:  Blazing  trails  in  the  organization' s white  spaces.  ” Star  performers  usually  go  above  and 
beyond  the  accepted  job  description  or  everyday  work  routines  to  offer  new,  bold  and  value-adding  ideas,  while 
average  performers  usually  think  about  ways  to  do  their  job  better  or  to  get  noticed  by  upper  management. 

2.  “Networking:  Knowing  who  knows  by  plugging  into  the  knowledge  network.  ” Star  performers  usually 
develop  pathway  to  knowledge  experts  who  can  help  them  complete  critical  path  tasks,  and  share  their  knowledge 
with  others  who  need  it.  Average  performers  usually  socialize  with  other  people  in  order  to  help  their  career. 

3.  “Self -management:  Managing  your  whole  life  at  work.  ” Star  performers  usually  create  opportunities, 
choose  work  which  contributes  to  critical  path  and  ensure  high  job  performance.  Average  performers,  however, 
focus  on  how  to  better  manage  their  time  and  projects. 

4.  “Perspective:  Getting  the  big  picture.  ” Star  performers  see  things  in  a larger  context  and  through  the 
various  viewpoints  of  customers,  competitors,  colleagues,  and  bosses.  Whereas  average  performers  see  things  only 
from  their  own  perspectives  and  make  sure  their  perspectives  are  valued. 

5.  “Followership:  Checking  your  ego  at  the  door  to  lead  in  assists.  ” Star  performers  help  organizations 
succeed  while  exercising  independent,  critical  thinking  on  goals,  tasks,  and  methods;  whereas  average  performers 
only  follow  the  guidelines  of  the  job  description. 

6.  “Teamwork:  Getting  real  about  teams.  ” Star  performers  share  with  team  members  the  group  goals, 
commitments,  work  activities,  schedules,  and  accomplishments  and  are  positive  contributors  to  group  dynamics. 
Average  performers,  however,  just  want  to  be  a part  of  the  team  and  work  cooperatively  with  others. 

7.  “Leadership:  Doing  small  “L”  leadership  in  a big  “L”  world.  ” Star  performers  use  their  expertise  and 
influence  to  get  their  colleagues  together  to  accomplish  important  tasks.  They  help  the  group  to  create  a clear  vision, 
find  the  resources  to  accomplish  the  task  and  lead  the  project  to  successful  completion.  Whereas  average  performers 
think  it  as  an  inborn  trait  that  allows  them  to  be  in  charge  and  have  the  power  to  make  important  decisions. 

8.  “Organizational  savvy:  Using  street  smarts  in  the  corporate  power  zone.  ” Star  performers  distinguish  it 
as  knowing  the  competing  interests  in  an  organization,  promoting  cooperation,  addressing  conflicts,  and 
accomplishing  tasks.  Average  performers  think  it  is  a talent  of  getting  themselves  noticed  by  the  right  people  and 
playing  office  politics. 

9.  “Show-and-tell:  Persuading  your  audience  with  the  right  message.  ” Star  performers  communicate  with 
others  in  the  most  effective  and  user-friendly  format  in  order  to  reach  and  persuade  their  audience.  Average 
performers  focus  on  getting  noticed  by  upper  management  through  their  own  image  and  message,  instead  of 
focusing  on  their  audience. 

Kelley  (1998;  1999a;  1999b)  suggested  that  for  average  performers  to  become  star  performers,  they  have  to 
change  the  ways  in  which  they  do  their  work  and  work  with  others.  The  implication  for  trainers/instructional 
designers  is  to  help  people  build  skills  in  these  nine  work  strategies  common  to  star  performers. 

Other  Strategies  Used  by  Star  Performers 

Hanson  (1993)  suggested  that  star  performers  are  believed  to  be  strong  communicators,  negotiators  and 
problem  solvers.  In  order  to  become  a star,  you  must  have  vision,  take  risks  and  have  an  entrepreneurial  spirit.  She 
summarized  some  strategies  used  by  star  performers  (Hanson,  1993,  p.  9): 

“Know  the  currency  of  your  environment.  ” Stars  always  attend  to  company  values,  and  follow  the  styles 
that  management  applauds. 


567 


“Remake  your  image.  ” What  makes  the  stars  different  from  the  average  employees  is  their  performance 
image.  Stars  are  committed  to  growing  instead  of  being  satisfied  with  their  existing  status.  They  often  ask  for 
feedback  from  their  superiors. 

“Visibility.  ” Star  performers  understand  the  importance  of  being  seen,  but  they  advertise  their 
accomplishments  without  bragging. 

“Cultivate  allies.  ” Star  performers  know  how  to  show  their  disagreement  in  a pleasant  way  and  how  to 
convert  critics  into  friends. 

“Continuing  education.  ” Star  performers  always  continue  to  develop  themselves  in  order  to  improve  their 
expertise  and  expand  their  knowledge. 

Sandberg’s  Interpretative  Approach  to  Competence 

A different  approach  to  examine  and  understand  human  competence  at  work,  called  an  interpretative 
approach,  was  proposed  by  Sandberg  (2000).  He  discussed  three  main  approaches  used  to  identify  competence: 
worker-oriented  approaches,  where  competence  is  defined  as  attributes  possessed  by  workers;  work  oriented 
approaches,  which  emphasizes  on  worker  attributes  that  are  strictly  work-related,  such  as  the  approach  took  by 
Spenser  and  Spenser  (1993);  and  multi-method-oriented  approaches,  which  suggest  competence  is  composed  of  a 
specific  set  of  attributes  (Sandberg,  2000).  He  suggested  these  three  approaches  are  essentially  based  on  the 
scientific  principles  of  the  rationalistic  research  tradition.  Sandberg  (2000)  argued  that  although  the  rationalistic 
approaches  contribute  to  our  understanding  of  competence,  the  view  of  competence  as  a set  of  attributes  may 
confirm  a researcher’s  own  model  of  competence  instead  of  capturing  the  workers’  competence.  In  addition,  the 
rationalistic  approaches  describe  human  competence  indirectly  and  view  it  as  composed  of  two  independent 
variables:  prerequisite  worker  attributes  and  work  activities. 

Sandberg  (2000)  then  proposed  interpretative  approaches  to  competence  which  views  competence  as 
“constituted  by  the  meaning  the  work  takes  on  for  the  worker  in  his  or  her  experience  of  it”  (p.  11).  Findings  of  his 
research  suggest  that  instead  of  a specific  set  of  attributes,  it  is  the  workers’  conception  of  their  work  that  organizes 
their  knowledge  and  skills  into  distinctive  competence  at  work.  The  workers’  conception  of  work  not  only  defines 
what  competence  he  or  she  develops  and  uses,  but  also  determines  the  hierarchy  of  competency  at  work.  The 
findings  challenge  the  traditional  view  of  competence  development  as  a linear  progression  from  novice  to  expert  and 
suggest  that  the  change  in  conceptions  of  work  is  more  fundamental  to  competence  development.  Sandberg  (2000) 
suggested,  instead  of  a specific  set  of  attributes  possessed  by  those  exemplary  performers,  it  is  the  variation  in 
workers’  perception  of  their  work  that  explains  why  some  people  perform  particular  work  better  than  others. 

Sandberg’s  (2000)  findings  suggest  a shift  of  focus  for  training  and  development  activities  from  attributes 
to  workers’  conception  of  their  work.  Instead  of  transferring  important  attributes  to  workers  who  don’t  have  them, 
training  and  development  should  focus  on  facilitating  the  development  of  competence  through  changing  conceptions 
of  work. 

Cognitive  Mechanisms  behind  Expert  Performance 

Hong  (1999)  suggested  that  cognitive  psychologists  have  been  investigating  expert  performance  for  a long 
time.  Previous  research  includes  the  studies  on  chess  expertise  (e.g.,  W.  G.  Chase  & Simon,  1973;  de  Groot,  1965) 
and  novice  versus  expert  performance  in  problem  solving  (e.g.,  Larkin,  McDermott,  Simon,  & Simon,  1980).  Recent 
research  includes  the  studies  on  the  role  of  deliberate  practice  in  the  acquisition  of  expertise  (e.g.,  Ericsson,  1996; 
Richman,  Gobet,  Staszewski,  & Simon,  1996).  These  studies  tended  to  investigate  the  cognitive  mechanisms  behind 
expert  performance.  The  expert  here  was  defined  as  “someone  who  exhibits  extremely  competent  performance  or 
falls  in  the  top  5%  of  performers  in  a domain;  someone  who  performs  at  the  level  of  an  experienced  professional” 
(Hong,  1999,  p.244).  Studies  conducted  by  cognitive  scientists  (e.g.,  Rabinowitz  & Glaser,  1985;  Sternberg,  1998) 
revealed  that  “individual  differences  of  cognitive  capacity  can  be  attributed  to  the  qualitative  and  quantitative 
differences  in  amount,  accessibility,  and  organization  of  knowledge,  mental  representations,  accuracy  and  speed  of 
information  processing,  and  efficiency  of  cognitive  strategies  and  metacognitive  skills”  (Hong,  1999,  p.244). 

Hong  (1999)  summarized  the  research  findings  on  expert  performance,  which  helped  to  identify  the 
underlying  cognitive  mechanisms  that  distinguish  experts  and  novices.  These  findings  suggested  four  typical 
cognitive  characteristics  that  distinguish  experts  from  novices  (p.  245-246): 

“ Knowledge  base  and  representations .”  This  refers  to  an  individuals’  domain  knowledge.  Research  studies 
have  shown  that  experts  have  better-organized,  superior  domain  knowledge.  The  knowledge  base  of  experts  also 
includes  extensive  representations  that  help  with  problem  solving,  the  solution  process  and  other  components 
(Richman  et  al.,  1996).  Chi,  Feltovich  and  Glaser  (1981)  suggested  experts  categorize  problems  based  on  basic 
representations  and  methods  relevant  to  solving  them,  while  novices  sort  problems  based  on  surface  features. 
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Ericsson  (1996)  suggested  experts  can  generate  a complex  representation  of  the  problem  and  integrate  it  with  their 
knowledge  to  select,  evaluate,  check  and  reason  about  alternative  actions. 

“Cognitive/perceptual  efficiency .”  Previous  studies  have  shown  experts  can  solve  problems  that  novices 
cannot  solve  or  that  experts  can  solve  them  more  rapidly  and  accurately.  Richman  et  al.  (1996)  suggested  the 
thinking  process  has  been  automated  so  that  they  cannot  solve  problems  without  conscious  attention. 

“Cognitive  strategies .”  Previous  cognitive  studies  on  novice  versus  expert  performance  in  problem  solving 
have  revealed  that  experts  and  novices  use  different  rules  and  strategies.  Experts  tend  to  work  forward  on  problems 
that  are  easy  for  them  and  work  backward  on  more  difficult  problems,  while  novices  tend  to  work  backward  from 
the  problem  goal  (Audet  & Abegg,  1996;  Larkin  et  al.,  1980;  Priest  & Lindsay,  1992;  Simon  & Simon,  1978). 

“Metacognition/S elf -regulated  learning .”  Metacognition  has  been  recognized  as  a key  component  which 
distinguishes  experts  from  novices  (e.g.,  Cheng,  1993;  Schwanenflugel,  Stevens,  & Carr,  1997).  Experts  have  better 
metacognitive  skills  such  as  planning,  questioning  and  solution  monitoring. 

Hong  (1999)  suggested  that  the  above  characteristics  may  be  required  to  explain  the  performance 
differences  between  experts  and  novices.  However,  how  these  characteristics  interact  with  each  other  and  what  are 
their  comparative  importance  to  the  performance  still  remain  unanswered. 

Implications  for  Trainers  and  Instructional  Designers 
Training  for  Excellence  or  Training  for  Mediocrity 

The  above  research  studies  have  very  important  implications  for  trainers  as  well  as  instructional  designers. 
They  help  to  answer  the  very  key  question  in  today’s  training  and  development:  do  we  train  people  to  reach  levels  of 
productivity  closer  to  that  of  the  exemplary  performers  or  do  we  train  people  to  reach  merely  minimum  performance 
standards? 

Zickefoose  (1979)  declared  that  we  should  stop  designing  training  that  produces  only  average  performers. 

In  order  to  stop  training  toward  mediocrity,  we  should  establish  curriculum  and  assessment  based  on  the  optimum 
performance.  Gilbert  (1978)  suggested  using  average  performance  as  the  standard  will  reduce  the  performer’s 
potential  for  improvement,  and  therefore  will  inevitably  lead  to  mediocrity.  On  the  other  hand,  using  exemplary 
performance  as  the  standard  suggests  all  performers  would  achieve  what  the  best  has  achieved.  Spenser  and  Spenser 
(1993)  concurred  with  Gilbert  and  suggested  organizations  should  use  the  characteristics  of  superior  performers  as 
their  standard  for  hiring  and  training  their  employees.  If  they  just  select  and  train  people  to  reach  the  current  average 
level  of  performance  it  will  be  essentially  mediocre. 

Chase  (1997)  pointed  out  that  most  training  programs  today  focus  on  improving  the  skills  of  inexperienced 
workers  to  meet  the  minimum  level  of  job  requirements  instead  of  teaching  them  how  to  excel.  She  quoted  what 
Dean  Spitzer,  the  senior  performance  consultant  in  IBM  Corp.,  said:  “We  tend  to  base  our  training  on  average 
performance,  on  mediocrity. . . rather  than  modeling  mediocrity,  let’s  use  a model  of  excellence.  As  long  as  we  have 
to  do  a job,  why  not  do  it  exceptionally?”  (p.  88).  In  today’s  increasingly  competitive  world,  training  for  excellence 
will  help  organizations  gain  competitive  advantage.  Therefore,  the  goal  of  trainers  and  instructional  designers  is  to 
train  for  excellence  instead  of  mediocrity  by  using  exemplary  performance  as  the  standard.  It  has  become  a recent 
trend  in  the  human  competency  movement. 

Who  are  the  Exemplars? 

Exemplars  are  also  called  exemplary  performers,  top  performers,  superior  performers,  expert  performers, 
outstanding  performers,  and  star  performers.  They  are  also  called  high  fliers  and  are  regarded  as  the  key  to  provide 
organizations  with  a competitive  edge  (Anonymous,  1995).  An  important  task  for  trainers  and  instructional 
designers  is  to  find  out  who  are  the  exemplars. 

Ellsworth  (2000)  pointed  out  that  exemplars  are  not  necessarily  the  best  performers  in  a particular  job, 
because  they  must  be  examples  whose  performance  that  others  can  emulate.  Therefore,  exemplars  are  not  genius 
with  inborn  abilities  that  no  one  else  has  or  workaholics  who  sacrifice  for  their  jobs  more  than  others.  They  are 
normal  people  who  have  achieved  the  best  practices  that  can  e duplicated  and  achieved  by  others. 

Froiland  (1993)  summarized  several  methods  to  identify  star  performers.  One  method,  suggested  by 
Harless,  was  to  first  analyze  the  company  goals  and  how  individual  accomplishments  contribute  to  fulfill  those 
goals.  He  also  distinguished  performance  from  knowledge  and  specified  that  star  performer  may  not  be  the  person 
who  knows  the  most.  Kelley  (1999b)  identified  star  performers  as  those  who  have  been  nominated  by  both  the 
managers  and  their  peers.  Another  method  was  employed  at  Du  Pont.  A group  of  “stellar”  employees  were  selected 
from  three  disciplines,  the  group  then  identified  key  competencies  possessed  by  star  performers  and  nominated  a 
group  of  workers  who  demonstrated  those  key  competencies  as  star  performers. 
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What  Makes  Them  Become  Stars? 

Once  the  star  performers  are  identified,  trainers  and  instructional  designers  need  to  find  out  what  makes  the 
star  performers  different  from  average  performers.  It  was  suggested  that  star  performers  have  “success  systems” 
which  help  them  perform  better  than  average  performers  (“How  everyone  else  can  learn,”  1976).  The  previous 
studies  (Gilbert,  1978;  Hong,  1999;  Kelley,  1998,  1999a,  1999b;  McClelland,  1973;  Sandberg,  2000;  Spenser  & 
Spenser,  1993)  have  examined  the  “success  systems”  and  provided  different  valuable  insights  on  the  reasons  of  the 
differences  between  exemplary  and  average  performers.  Trainers  and  instructional  designers  can  use  the  results  of 
these  studies  as  a foundation  to  train  and  develop  people  to  reach  the  level  of  productivity  closer  to  that  of  the 
exemplars. 

Competency-Based  Training  Versus  Traditional  Training 

The  competency-based  training  approach  has  been  the  current  trend  in  training  and  development.  It  focuses 
attention  on  training  or  developing  for  “best-in-class”  performers,  trying  to  build  individual  competence  to  reach  the 
level  of  exemplary  performance  specified  in  the  competency  models.  The  competency-based  training  approach 
requires  trainers  to  focus  on  the  conditions  essential  for  exemplary  performance  instead  of  minimum  performance 
requirements  (Dubois  & Rothwell,  2004). 

The  traditional  training  approach  typically  involves  conducting  a training  needs  assessment  to  identify  what 
workers  must  know,  do  or  feel  to  perform  their  job  successfully,  and  then  identify  the  gap  between  the  performance 
requirements  and  what  they  actually  know,  do  or  feel.  The  purpose  of  training  is  to  close  the  gap  in  order  to  reach 
the  minimum  performance  requirements.  The  competency-based  training,  however,  focused  on  building 
competencies  beyond  knowledge,  skills  and  attitudes  to  include  other  personal  characteristics,  such  as  motivation, 
personal  traits,  that  may  distinguish  exemplary  from  merely  successfully  performers.  The  purpose  of  training  is  to 
train  and  develop  people  to  reach  the  level  of  productivity  closer  to  that  of  the  exemplary  performers.  The  key 
questions  trainers  ask  are:  what  are  the  essential  working  conditions  for  exemplars?  What  competencies  must  people 
process  in  order  to  rival  the  accomplishments  achieved  by  the  exemplars?  Who  is  the  audience  of  the  training,  and 
what  reasonable  assumptions  can  be  made  about  how  close  those  individual’s  characteristics  are  to  that  of  the 
exemplars?  (Dubois  & Rothwell,  2004). 

Dubois  and  Rothwell  (2004)  suggested  that  compared  to  the  traditional  training  approach,  the  competency- 
based  training  approach  has  the  advantages  of  describing  how  exemplary  performers  achieve  their  success  and 
recognizing  that  there  are  dimensions  to  performance  beyond  knowledge,  skills  and  attitudes.  It  is  a more  holistic 
approach.  They  also  pointed  out  it  is  important  to  focus  attention  on  training  to  build  individual  competence.  First  of 
all,  it’s  highly  individualized  to  meet  individual  learners’  needs.  Secondly,  competencies  provided  increased  focus 
on  learning  objectives  and  performance  expectations.  Thirdly,  it  allows  learners  to  build  competencies  in  a 
meaningful  way  to  themselves.  Fourthly,  learners  can  identify  many  useful  learning  resources  in  different  settings. 
Finally,  this  approach  is  practical  when  training  is  valuable  in  achieving  organizational  success. 

Strategic  Systems  Model  (SSM)  and  Competency  Modeling 

The  competency-based  training  approach  suggests  that  trainers  and  instructional  designers  should 
reexamine  the  traditional  Instructional  Systems  Design  (ISD)  model  to  focus  on  competence-based  training  and 
development.  The  traditional  ISD  model  is  a systematic  approach  to  training  and  instructional  design  and 
development.  It  is  also  called  SAT  (System  Approach  to  Training)  or  ADDIE  (Analysis,  Design,  Development, 
Implementation,  Evaluation)  (Clark,  1995).  Dubois  (1993)  provided  a Strategic  Systems  Model  (SSM)  by 
reinventing  the  traditional  ISD  model.  The  SSM  focuses  on  creating  and  evaluating  competence-based  employee 
training  and  other  performance  improvement  experiences. 

The  SSM  adds  the  development  of  the  competency  models  to  define  the  worker’s  competencies.  Rothwell 
(2002)  pointed  out  competency  modeling  focuses  on  discovering  the  differences  between  the  exemplary  and  average 
performers  in  the  same  job  category.  By  using  competency  models  as  a foundation  for  training  and  development, 
trainers  and  instructional  designers  can  recognize  other  alternatives  for  competency  building  than  training  alone 
(Dubois  & Rothwell,  2004). 

Spenser  and  Spenser  (1993)  also  suggested  organizations  should  use  competency  models  which  specify  the 
competencies  of  the  superior  performers  in  the  decision-making  process  of  recruiting,  placement,  retention  and 
promotion.  These  decisions  are  made  by  matching  between  the  job  competency  requirements  and  person 
competencies.  Competence-based  selection  has  several  advantages:  help  organizations  gain  competitive  advantage, 
decrease  turnover  rate,  identify  people  who  have  the  potential  to  be  promoted,  greatly  cut  the  new  hire  learning 
curve  periods  and  help  determine  training  needs  at  entry.  During  the  recruiting  process,  organizations  should  focus 
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on  core  competencies  that  are  more  likely  to  predict  candidates’  long  term  career  success  and  are  more  difficult  to 
develop  through  training;  those  core  competencies  are  more  cost-effective  to  select  than  to  develop. 

SSM  and  competence  modeling  has  been  widely  accepted  by  trainers  and  instructional  designers  to 
establish  competency-based  employee  training  and  other  performance  improvement  requirements  in  organization 
settings  (Dubois  & Rothwell,  2004). 
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